-- John Tulp <johnt...@tulpex.com> tulpex
On Sun, 2022-12-11 at 00:05 -0800, Doug Hardie wrote: > > On Dec 10, 2022, at 11:09 AM, John Tulp <johnt...@tulpholdings.com> wrote: > > > > > > > > On Fri, 2022-12-09 at 20:03 -0800, Doug Hardie wrote: > >>> On Dec 4, 2022, at 1:42 PM, Doug Hardie <bc...@lafn.org> wrote: > >>> > >>>> On Dec 3, 2022, at 11:50 PM, Doug Hardie <bc...@lafn.org> wrote: > >>>> > >>>> I started to investigate using doveadm backup to backup my mail > >>>> system. I have a small number of users and the mail store is not > >>>> large. It uses maildir format. I setup a test system that is not > >>>> connected to the internet and started up dovecot. I used the > >>>> following command to backup one user: > >>>> > >>>> doveadm backup -u ben remote:test > >>>> > >>>> ben is the user is in the mail store. Test is the actual server > >>>> name. That worked just fine. The maildir was copied completely > >>>> (as best as I can tell with ls). Then I tried the second user: > >>>> > >>>> doveadm backup -u jean remote:test > >>>> > >>>> This gives 2 error messages: > >>>> > >>>> doveadm(jean)[]<McRYOFI0jGN1MwAAmC+70w>: Error: Mailbox INBOX: > >>>> Failed to get attribute > >>>> vendor/vendor.dovecot/pvt/server/sieve/files/.dovecot: Mailbox > >>>> attributes not enabled > >>>> > >>>> doveadm(jean)[]<0IwxIlI0jGMgUwAAZU03Dg>: Error: Remote command > >>>> returned error 65: ssh test doveadm dsync-server -ujean -U > >>>> > >>>> In addition, the maildir directories are created, but there are no > >>>> emails in any of them (e.g., cur). What is the problem with the > >>>> 2nd and why does it behave differently from the first? > >>> > >>> I managed to resolve most of the issue. I use pigeonhole on the > >>> primary server. Apparently not having pigeonhole installed on the > >>> test machine caused the errors above. The test machine was never > >>> intended to receive mail hence, no need to install pigeonhole as the > >>> LDA would never be used. However, when the backup was running, it > >>> choked on transferring the sieve file. I have no idea where the > >>> mentioned file resides as I couldn't find it anywhere on the primary > >>> server. Installing pigeonhole resolved the issue for all but one > >>> user. With that user, I get the following error messages: > >>> > >>> doveadm(doug)[]<NAfSLTASjWNWTAAAmC+70w>: Panic: file > >>> istream-seekable.c: line 238 (read_from_buffer): assertion failed: > >>> (*ret_r > 0) > >>> Abort > >>> > >>> doveadm(doug)[]<C2AvHjASjWN2gQAAZU03Dg>: Error: read(test) failed: > >>> EOF (last sent=mail, last recv=mail_request (EOL)) > >>> > >>> doveadm(doug)[]<C2AvHjASjWN2gQAAZU03Dg>: Error: Remote command > >>> returned error 134: ssh test doveadm dsync-server -udoug -U > >>> > >>> In addition, only a few cur files are transferred in INBOX. > >>> Repeating the backup generates the same errors and no additional > >>> emails are transferred. I am wondering if the problem is something > >>> in the INBOX. > >> > >> > >> > >> > >> I have pretty much reached a dead end. I am unable to determine the > >> cause of this abnormal termination. The logged messages don't give > >> much help. I can't tell if it is the primary server or test machine > >> that is terminating ssh. I have setup a second test machine. Both of > >> the test machines are raspberry pi 4s. The real backup machine is > >> intel. On both the test machines, 2 of the three users are backedup > >> properly with no errors. Only one use has the issue. It builds the > >> directory structure correctly then starts transferring the INBOX cur > >> files. Eight files are transferred correctly before it stop. It is > >> the same set of 8 on both test machines. That leads me to believe > >> there is something funny in one of the messages, but which one. There > >> are over 100 messages in that directory. The order of file transfer > >> is not by date. > >> > >> > >> How do I get doveadm to tell me which file is being transferred when > >> the problem occurs? > > > > if the problem is in one of the messages, perhaps change the test set. > > for convenience, can temporarily rename/move things to get them out of > > the way. divide test set in half, then in half again, etc., to zero in > > on the one causing the issue, that'll show if a particular message is > > problem or not. > > > > once you find a problem message, if the why isn't obvious, i'd try > > looking at it in hex, check permissions, etc. > > I have found the cause for the errors: Somehow I had overlooked the -D > argument to doveadm. That is really helpful in diagnosing backup issues. I > discovered that there was one email that was 130+ MB. Doveadm cannot handle > that. It appears there is a message size limit somewhere. I don't know if > that is changeable or not. Anyway, removing that email from the mailbox, > deleting it from the Trash, and purging the account enabled backup to work > properly. I don't get too many emails that large, but it does happen > occasionally. How do I go about telling dovecot to handle those? > > -- Doug > > sorry, no clue. i did find this link quickly... https://doc.dovecot.org/configuration_manual/quota/ looks like the following section near the bottom of page may apply ? "Maximum Saved Mail Size" again, sorry, email is not "my thing" (chuckle inserted here).
