On Mon, Jan 03, 2005 at 05:39:19PM +0100, Dag Wieers wrote:
> With newer rsyncs, rsync seems to sort the transaction set, so I
> cannot longer use this trick to have the metadata uploaded just after
> the data in the same transaction set.
Rsync has always sorted the list of files to be sent, so this is not
something that is different between 2.5.x and 2.6.x. I'd be interested
in hearing what you believe to be different in how files are processed.
> I was wondering if it was possible and acceptable to have an rsync
> option to update the whole transaction in a atomic (or near-atomic
> way).
One way to do this would be to use the --link-dest option to create a
new hierarchy of files (with only the changed files getting sent, and
all unchanged files being hard-linked to the prior files) and then
moving the whole file-set into place all at once. Imagine that there
is a hierarchy you want to update in /dest/cur by running this script:
[ -d /dest/old ] && mv /dest/old /dest/new
rsync -av --link-dest=/dest/cur host:/src/ /dest/new &&
mv /dest/cur /dest/old &&
mv /dest/new /dest
This sequence allows any currently-running rsync processes to finish
their grabbing of files because they're still running through the files
that got put into /dest/old. As long as you don't do this rotation very
rapidly (i.e. the time between two updates must be longer than the
runtime of your longest-running downloader), it will all work fine. The
above also works right if rsync errors-out during a transfer (i.e., it
continues to update the "new" dir, and then moves it into place).
For a push, if you're using a remote-shell connection, you cold create
a script that does the same thing as the above and run it remotely via
rsync. First, the script (which I've named "bin/daily-sync" in the
receiving user's home dir):
#!/bin/sh
[ -d /dest/old ] && mv /dest/old /dest/new
rsync "$@" &&
mv /dest/cur /dest/old &&
mv /dest/new /dest
Then, run this command from the pushing system:
rsync -av --rsync-path=bin/daily-sync --link-dest=/dest/cur /src/
host:/dest/new
The only difficult case where I don't have a good solution is for
pushing files via daemon mode.
..wayne..
--
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
