From: Demitri Muna
Subject: Re: [infrastructure 14923] Re: [sloan-25mlog 3133] Re: 2.5m obslog 20161002 (SJD 57664)
Date: Mon, 3 Oct 2016 16:57:06 -0400
--Apple-Mail=_44D28648-F4E2-4330-BC88-40EF724A0545
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="us-ascii"
Hi,
I've written a fair amount about this in the past. It seems wasteful =
(time, disk space) to perform full backups of data that is accumulating =
(i.e. older data not changing) rather than broadly changing. Is the =
APOGEE data timestamped in any way? Rather than a full dump, one could =
perform backups in time ranges. One would have to look at the data =
model, but it seems to me that making incremental backups where =
appropriate would make more sense. Full backups can be made monthly =
during engineering time.
Is the source data always available or moved off-site? If it remains =
local, it can be considered the backup and reimported if a restoration =
is needed.
I'm sure there are several solutions - the main point is not to =
repeatedly perform expensive backups of data that doesn't change.
Cheers,
Demitri
_________________________________________
Demitri Muna
http://muna.com
Center for Cosmology and AstroParticle Physics
& Department of Astronomy
Le Ohio State University
My Projects:
http://nightlightapp.io
http://trillianverse.org
http://scicoder.org
--Apple-Mail=_44D28648-F4E2-4330-BC88-40EF724A0545
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html; charset="us-ascii"
Hi,I've =
written a fair amount about this in the past. It seems wasteful (time, =
disk space) to perform full backups of data that is accumulating (i.e. =
older data not changing) rather than broadly changing. Is the APOGEE =
data timestamped in any way? Rather than a full dump, one could perform =
backups in time ranges. One would have to look at the data model, but it =
seems to me that making incremental backups where appropriate would make =
more sense. Full backups can be made monthly during engineering =
time.Is the =
source data always available or moved off-site? If it remains local, it =
can be considered the backup and reimported if a restoration is =
needed.I'm =
sure there are several solutions - the main point is not to repeatedly =
perform expensive backups of data that doesn't change.Cheers,Demitri
_________________________________________Demitri=
 Munahttp://muna.comCenter for Cosmology and AstroParticle =
Physics& Department of AstronomyLe Ohio State =
UniversityMy Projects:http://nightlightapp.iohttp://trillianverse.orghttp://scicoder.org
=
--Apple-Mail=_44D28648-F4E2-4330-BC88-40EF724A0545--