-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 - --inplace and --append-verify are essentially irrelevant when - --link-dest is in play. With --link-dest in play the target system must write an entirely new file even for a change in permissions or timestamps so any potential benefit by these options are out the window from the start. The only thing they can do is add the possibility of incomplete or corrupt copies on the target.
On 12/25/2014 09:49 PM, Askar Safin wrote: > Hi. This is bug report and simultaneously urgent asking for help. I > am trying to write my rsync wrapper script, which will create > minutely snapshots of my data using --link-dest. I want this script > to be robust, it should work even if I do suspend/hibernate/reboot > without notifying the script about this actions, it should work if > I make hard-reset of the computer and if I disconnect network. Also > this script should transfer large files with unstable network link. > I. e. I want the following: I connected internet, it transferred > some part of my large file, I disconnected inet, then I connected > again, the script continued transferring. So, I wrote the following > script (this script should be run locally, i. e. on my home > computer with my data): > > set +e > > while :; do if rsync --archive --copy-unsafe-links --no-owner > --no-group --one-file-system --compress --progress --inplace > --append-verify --partial --delete-after --fuzzy --timeout=60 > --link-dest=../latest-complete "$DIR/sync" > "$HOST":backups/in-progress; then TIME="$(TZ=UTC date +%F-%H%M%S)" > > ssh "$HOST" "mv -T ~/backups/in-progress ~/backups/$TIME && ln -sfn > $TIME ~/backups/latest-complete" || : fi > > sleep 60 done > > It seems for me that this script address all this issues and it is > OK, right? > > As I said, I want the script to work with large files with unstable > connection, so I decided to use "--inplace --append-verify" option. > Am I right? Or just "--partial --append-verify" without "--inplace" > will go? How exactly "--partial" without "--inplace" works? Where > partial data is stored, in what files? How rsync knows it should > continue transfer? Will --partial work in case of suddenly local > computer hard reset? > > Recently I noticed the following: I have in > "$HOST":backups/2014-11-22-003120 too new copy of some my file. I > am sure. Also, this file has a lot of hard links. So, I think the > following happened: at 2014-11-22 actual right copy of this file > was created at backup host. Then, some more snapshots was created > with hardlinks to this file. Then, rsync started creating new > snapshot in "in-progress", it created one more hardlink to that > file and then failed for some reason (reboot, inet disconnection, > etc). Then the file was changed, and rsync started again. rsync > continued to update "in-progress" dir. It noticed that the file > changed and updated it *without unlinking it*. So, all hardlinks, > including very old ones, was updated. And this was wrong. > > So, what to do? I. e. how to fix the script? Or maybe this is a bug > in rsync? I. e. "--link-dest --inplace doesn't unlink"? Some quick > fix will be good. > > Maybe I should remove --inplace and leave --partial? But will the > script still be such robust in this case? == Askar Safin > http://vk.com/safinaskar Kazan, Russia > - -- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. ke...@futurequest.net (work) Orlando, Florida k...@sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~ -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAlSc0L4ACgkQVKC1jlbQAQcm7wCgqHNKL2pPMjTkBgatFYrhhV8i Al4AoM2N/YVwi2nVWhRkwDDWvcBPw5le =bDmb -----END PGP SIGNATURE----- -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
