I agree that the Tower of Hanoi distribution is a good method.
It ensures a better depth. And with 10 copies, it would really, really
be good depth.
With accounting systems though, it's good to have a backup of the data
just -before- they close the fiscal period...
(the call we get is -- OMG we forgot to post that $250K before we closed
-- admin is going to kill us -- can we go back?)
We've only had a couple of connectivity issues with 2 or 3 clients over
4 years that caused it to miss one of the monthly rotations.
(the auto-retry off business hours is what has increased the resiliency
of the process at sites with less than perfect internet service)
--
Larry Irwin
Email: lrir...@alum.wustl.edu
On 06/22/2016 09:20 PM, Bell, Robert (CSIRO IM&T, Docklands) wrote:
Folks,
Am 20.06.2016 um 22:01 schrieb Larry Irwin (gmail):
> The scripts I use analyze the rsync log after it completes and then
sftp's a summary to the root of the just completed rsync.
> If no summary is found or the summary is that it failed, the folder
rotation for that set is skipped and that folder is re-used on the
subsequent rsync.
> The key here is that the folder rotation script runs separately
from the rsync script(s).
That is what we found to be important some years ago - do most of the
management outside of the scripts that do rsync.
In particular, our scripts prepare a target backup directory with a
name of the form:
slash.20151122.seq.1379.current
- the name of the area being backed-up is obvious, as is the date.
We use Tower of Hanoi management (see the update at
https://en.wikipedia.org/wiki/Backup_rotation_scheme#Tower_of_Hanoi),
so embed a sequence number in the backup directory name, and the
suffix .current to indicate a current backup target.
we can keep on trying the rsync backups until we get success, and the
scripts then remove the .current suffix. This works because the rsync
includes the --delete option, used since the backup target can be a
recycled hard-linked directory: also recommended.
> For each entity I want to rsync, I create a named folder to
identify it and the rsync'd data is held in sub-folders:
> daily.[1-7] and monthly.[1-3]
> When I rsync, I rsync into daily.0 using daily.1 as the link-dest.
> Then the rotation script checks daily.0/rsync.summary - and if it
worked, it removes daily.7 and renames the daily folders.
> On the first of the month, the rotation script removes monthly.3,
renames the other 2 and makes a complete hard-link copy of daily.1 to
monthly.1
> It's been running now for about 4 years and, in my environment, the
10 copies take about 4 times the space of a single copy.
> (we do complete copies of linux servers - starting from /)
> If there's a good spot to post the scripts, I'd be glad to put them
up.
I'd recommend Tower of Hanoi, as referenced above, for managing sets
of backups - far simpler than dealing with days and dates, etc. For
example, the above mentions special actions on the first of the month
- what if that is missed because of some failure - more special case
scripting? Tower of Hanoi is self-healing....
Regards
Rob.
Dr Robert C. Bell
HPC National Partnerships | Scientific Computing
CSIRO Information Management and Technology
T +61 3 9545 2368 (Docklands and Clayton)
T +61 3 9669 8102 (Docklands only)
Mob +61 428 108 333
robert.b...@csiro.au<mailto:robert.b...@csiro.au> | www.csiro.au
https://wiki.csiro.au/display/ASC/Scientific+Computing+Homepage
Also: Bureau of Meteorology Scientific Computing Services
Street: CSIRO SC Lvl 11, 700 Collins Street, Docklands Vic 3008,
Australia
Postal: CSIRO SC Lvl 11, GPO Box 1289, Melbourne Vic 3001, Australia
Street: CSIRO IMT, Rivett Room RNW 2.03, Clayton Vic 3168, Australia
Postal: CSIRO IMT, Private Bag 10, Clayton South Vic 3169, Australia
Parcels: CSIRO IMT, Gate 3 Normanby Road, Clayton Vic 3168, Australia
PLEASE NOTE
The information contained in this email may be confidential or
privileged. Any unauthorised use or disclosure is prohibited. If
you have received this email in error, please delete it immediately
and notify the sender by return email. Thank you. To the extent
permitted by law, CSIRO does not represent, warrant and/or
guarantee that the integrity of this communication has been
maintained or that the communication is free of errors, virus,
interception or interference.
Please consider the environment before printing this email.
On 22/06/2016 10:00 pm, rsync-requ...@lists.samba.org wrote:
Send rsync mailing list submissions to
rsync@lists.samba.org
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.samba.org/mailman/listinfo/rsync
or, via email, send a message with subject or body 'help' to
rsync-requ...@lists.samba.org
You can reach the person managing the list at
rsync-ow...@lists.samba.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of rsync digest..."
To unsubscribe or change options:
https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
---------------------------------------
Today's Topics:
1. Re: rsync script for snapshot backups (Dennis Steinkamp)
2. Re: rsync script for snapshot backups (Petros Angelatos)
_______________________________________
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
rsync mailing list
rsync@lists.samba.org
https://lists.samba.org/mailman/listinfo/rsync
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
