[EMAIL PROTECTED] wrote: > > Rsync seems to be loading information about each file into memory before > comparing the lists of files and doing the actual transfer. > That may be a lot of memory if you have a lot of files. > > I sometimes overcome this by rsync’ing each user or domain one at a time. > That way you will also limit issues of files no longer existing once the > transfer begins (makes rsync generate errors). >
If you are using rsync2 then definitely this is good advice. Massive memory consumption to backup large mailboxes (and a long time before anything starts happening, ie snapshot useful) However, with rsync3 you should look at the options required to do use the incremental protocol. This trades a bit of efficiency on hardlinked files for lower memory and perhaps faster sync speeds. I haven't personally tried this, but reports on the web seem promising. You need rsync3 at both ends of the link and to examine your sync options a little However, one thing which is sadly missing on rsync is a fuzzy option which can spot files moving from /new to /cur... This may well cause additional load for imap backups which is potentially avoidable with a simple copy. I suspect it would be easy to patch a custom bit of code to handle this though..? > One option that I would prefer if I were to backup the entire store with > one command would be generating a snapshot of the file system. > And then rsync or cp that snapshot. That way you’ll always get a > consistent backup and you won’t have to worry about how long the backup > takes to finish. > Snapshot seems like an excellent idea to avoid files missing files moving between /cur and /new. However, it should be pointed out that this is extra io for the server (with LVM at least) whilst the backup is running I should think rsync3 incremental, plus some custom patching to look for files moving between /cur and /new would be very efficient for backing up maildir filestores (at least to the extent your filesystem allows efficient iterating over lots of files) Ed W
