On 19 June 2016 at 10:27, Simon Hobson <li...@thehobsons.co.uk> wrote:
> Dennis Steinkamp <den...@lightandshadow.tv> wrote:
>
>> i tried to create a simple rsync script that should create daily backups
>> from a ZFS storage and put them into a timestamp folder.
>> After creating the initial full backup, the following backups should only
>> contain "new data" and the rest will be referenced via hardlinks (-link-dest)
>> ...
>> Well, it works but there is a huge flaw with his approach and i am not able
>> to solve it on my own unfortunately.
>> As long as the backups are finishing properly, everything is fine but as
>> soon as one backup job couldn`t be finished for some reason, (like it will
>> be aborted accidently or a power cut occurs)
>> the whole backup chain is messed up and usually the script creates a new
>> full backup which fills up my backup storage.
>
> Yes indeed, this is a typical flaw with many systems - you often need to
> throw away the partial backup.
> One option that comes to mind is this :
> Create the new backup in a directory called (for example) "new" or
> "in-progress". If, and only if, the backup completes, then rename this to a
> timestamp. If when you start a new backup, if the in-progress folder exists,
> then use that and it'll be freshened to the current source state.
I have an extremely similar script for my backups and that's exactly
what I do to deal with backups that are stopped mid-way, either by
power failures or by me. I rsync to a .tmp-$target directory, where
$target is what I'm backing up. I have separate backups for my rootfs
and /home. I also start the whole thing under ionice so that my
computer doesn't get slow from all this I/O. Lastly, before renaming
the .tmp-$target to the final directory I do a `sync -f` because rsync
doesn't seem to call fsync() when copying files and you can have a
failed backup if a power failure happens after the rename().
Here is my script:
#!/bin/bash
set -o errexit
set -o pipefail
target=$1
case "$target" in
home)
source=/home
;;
root)
source=/
;;
esac
PATHTOBACKUP=/root/backup
date=$(date --utc "+%Y-%m-%dT%H:%M:%S")
ionice --class 3 rsync \
--archive \
--verbose \
--one-file-system \
--sparse \
--delete \
--compress \
--log-file=$PATHTOBACKUP/.tmp-$target.log \
--link-dest=$PATHTOBACKUP/$target-current \
$source $PATHTOBACKUP/.tmp-$target
sync -f $PATHTOBACKUP/.tmp-$target
mv $PATHTOBACKUP/.tmp-$target.log $PATHTOBACKUP/$target-$date.log
mv $PATHTOBACKUP/.tmp-$target $PATHTOBACKUP/$target-$date
ln --symbolic --force --no-dereference $target-$date
$PATHTOBACKUP/$target-current
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
