On Tue, Feb 5, 2013 at 10:41 PM, Joe Beaubien <joe.beaub...@gmail.com>wrote:
> > > > On Mon, Feb 4, 2013 at 9:01 PM, Timo Sirainen <t...@iki.fi> wrote: > >> On Mon, 2013-02-04 at 00:57 +0000, Ben Morrow wrote: >> > I can't give authoratitive answers to either of these, but... >> > >> > At 6PM -0500 on 3/02/13 you (Joe Beaubien) wrote: >> > > >> > > I'm currently trying to setup remote backups of my emails but i'm >> running >> > > into issues (mdbox format, indexes and storage in the same folder >> > > hierarchy). >> > > >> > > Local backup command: dsync -u "my_user" backup /backups/my_user >> > > >> > > (1) Recently, I noticed that the local backup takes up twice the size >> as >> > > the original mail location (8gb vs. 4gb). I purged alot of emails >> from the >> > > original location, so the size shrunk, but the local backup just >> keeps on >> > > getting bigger. I couldn't find any dsync option that would delete >> extra >> > > emails. >> > > >> > > - Question: Why isn't the local backup synced properly and remove the >> extra >> > > emails? >> > >> > Are you running 'doveadm purge' on the backed-up dbox? It looks to me as >> > though dsync doesn't do that. I don't know if there's any (simple) way >> > to do that without a running Dovecot instance attached the dbox >> > directory: it's not entirely clear to me whether doveadm will run >> > locally without contacting a doveadm-server instance running under >> > Dovecot, nor how to point 'doveadm purge' at an arbitrary directory. >> >> Right. doveadm -o mail=mdbox:/backups/my_user purge >> > > This worked (at least it seems to, the source and destination are roughly > the same size).. > > However, if the the original email location has already been purge, does > the backup email location also need to be purged? > > >> > It might be easier to dsync to a Maildir instead. This should preserve >> > all the Dovecot-visible metadata, and dsyncing back to the dbox for >> > restore should put it all back. >> >> Better sdbox than maildir. >> > > I'd rather stick to mdbox for my remote backups. I have a single email > account with over 1.5 million emails in it. With a 1-file-per-message > storage, this would be slow/hell to run. Unless there is a better way. > > >> >> > > (2) What is the best why to copy this local backup to a remote >> location >> > > that does NOT have the possibility to run dsync. >> > > >> > > - Question 1: is rsync safe to use and will this data work for >> restore? >> > > >> > > - Question 2: Would it be safe to simply rsync the original >> mail_location >> > > to the remote server? >> > >> > AFAICT, if Dovecot is stopped on both sides of the transfer it should be >> > safe. If either side has a currently running Dovecot instance (or any >> > other Dovecot tools, like a concurrent dsync) using that dbox, it's >> > likely rsync will copy an inconsistent snapshot of the data, and the >> > result will be corrupted. >> >> It won't be badly corrupted though. At worst Dovecot will rebuild the >> indexes, which takes a while. And most users probably won't get any >> corruption at all. >> >> >> > I think there was a misunderstanding of the setup. dsync is only running > on the local side, the remote side is a "dumb" rsync server that I don't > fully control. > > > what i was trying to ask with my last question is the following: I'm trying to do remote backups of an mdbox setup, and considering that I only have dsync running on the local side (not on the remote side), is it safe to simply do an rsync of the mail_location to the remote server, or should I first do a dsync (make a local duplicate) and then rsync the duplcate out to the remote server? (wish i had a whiteboard right about now). Thanks, -Joe
