>>>>> " " == August Zajonc <[EMAIL PROTECTED]> writes:
> Thanks for some interesting pointers. Unless I'm missing
> something it seems that rsync will still have to go through
> each filename and see if it matches --include. We're kinda
> pushing it here, but have around 500,000 files. Files need to
> be synced at the longest once a minute, and my experiance so
> far is that simply building a large filelist (rsync appears to
> do its entire search and *then* transfer) is relativly
> timeconsuming, even if few of the files match.
> What has worked as a stop gap measure is to pass the filelist
> as a command line option such as rsync 'cat filelist' dest::etc
> My concern was that there would be some limitation there, but
> we've managed all right with a small 800 file test. My feeling
> is that if rsync accepts a whole set of files on the command
> line it would be relativly straightforward to have it build a
> filelist from, well, a filelist. I'm gonna have one of the guys
> here take a look at patching it to some behavior like that.
> August
You probably want to push any changes to all clients instead of all
clients checking weather changes appeared.
So the simplest way would be to check for files with newer access time
and then rsync them to the clients. So once a minute you run a simple
find over the 500,000 files locally at the server, which will stress
the cache, so have a lot of ram. :)
In case something gone missing (like when a maschine was down or
network problems or if a file was deleted), do a full rsync once an
hour or once a day.
Alternatively to that, why not use afs or coda to realtime mirror the
complete filesystem or write some small stubs for xfs that will rsync
any changed files.
MfG
Goswin
