Andrew Gideon wrote:
I currently do incremental backups using --link-dest. Unchanged files
are hard links to the previous snapshot; changed files are new copies.
Where this "fails" is for large files that have received small changes.
The directory containing my main IMAP account, for example, typically
generates between 1 and 2 G of daily backup data as I file messages in my
inbox. Yesterday, though, I filed some old messages in my inbox. This
touched files I don't typically touch. They were minor changes in the
scheme of things, but the result was an 11G backup of that volume last
night.
If you resist using a fileformat which has some kind of implicit
"chunking" capability (eg maildir isn't perfect, but kind of goes in the
opposite direction and chunks your mail into lots of smaller files -
perhaps too many in the opinion of some...), then you really need a
backup system which can "chunk" your backup files and only store the
differences
There are a few like rsnapshot which might suit your needs. Or if you
want something different then perhaps try brackup, which is kind of
rsync like, but has a "chunking" phase built into it which tries to be
rather smart about dividing files up into the hopefully static bit and
the variable bit. eg in the case of MP3s it would create one chunk for
the data and one chunk for the tagging, that way you can retag your mp3
collection and it won't re-upload the whole lot
Ideally I would like to see a kind of half and half maildir/mbox format
emerge for email (perhaps dbox from dovecot will get there?). The idea
is that it would use mbox for storage, but break the files up into
chunks of about a couple of MB for each mbox file. This way you would
have much smaller files to defrag in the case of deletes, but you would
gain the packing efficiency of mbox (especially if you use compressed
mailboxes, eg dovecot)
Good luck
Ed W
--
Please use reply-all for most replies to avoid omitting the mailing list.
To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
