Hi. This is bug report and simultaneously urgent asking for help. I am trying
to write my rsync wrapper script, which will create minutely snapshots of my
data using --link-dest. I want this script to be robust, it should work even if
I do suspend/hibernate/reboot without notifying the script about this actions,
it should work if I make hard-reset of the computer and if I disconnect
network. Also this script should transfer large files with unstable network
link. I. e. I want the following: I connected internet, it transferred some
part of my large file, I disconnected inet, then I connected again, the script
continued transferring. So, I wrote the following script (this script should be
run locally, i. e. on my home computer with my data):
set +e
while :; do
if rsync --archive --copy-unsafe-links --no-owner --no-group
--one-file-system --compress --progress --inplace --append-verify --partial
--delete-after --fuzzy --timeout=60 --link-dest=../latest-complete "$DIR/sync"
"$HOST":backups/in-progress; then
TIME="$(TZ=UTC date +%F-%H%M%S)"
ssh "$HOST" "mv -T ~/backups/in-progress ~/backups/$TIME && ln
-sfn $TIME ~/backups/latest-complete" || :
fi
sleep 60
done
It seems for me that this script address all this issues and it is OK, right?
As I said, I want the script to work with large files with unstable connection,
so I decided to use "--inplace --append-verify" option. Am I right? Or just
"--partial --append-verify" without "--inplace" will go? How exactly
"--partial" without "--inplace" works? Where partial data is stored, in what
files? How rsync knows it should continue transfer? Will --partial work in case
of suddenly local computer hard reset?
Recently I noticed the following: I have in "$HOST":backups/2014-11-22-003120
too new copy of some my file. I am sure. Also, this file has a lot of hard
links. So, I think the following happened: at 2014-11-22 actual right copy of
this file was created at backup host. Then, some more snapshots was created
with hardlinks to this file. Then, rsync started creating new snapshot in
"in-progress", it created one more hardlink to that file and then failed for
some reason (reboot, inet disconnection, etc). Then the file was changed, and
rsync started again. rsync continued to update "in-progress" dir. It noticed
that the file changed and updated it *without unlinking it*. So, all hardlinks,
including very old ones, was updated. And this was wrong.
So, what to do? I. e. how to fix the script? Or maybe this is a bug in rsync?
I. e. "--link-dest --inplace doesn't unlink"? Some quick fix will be good.
Maybe I should remove --inplace and leave --partial? But will the script still
be such robust in this case?
==
Askar Safin
http://vk.com/safinaskar
Kazan, Russia
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
