Hi Wietse, et all.
Not sure whether this is known already or not,... in any case I think it's quite critical.. I recently stumbled over several MUAs/tools (e.g. Evolution, getmail) that have their problems with the mbox format, namely by corruption stored or imported mail in not quoting From_ lines correctly... or by intentionally using the mboxo format, which inherently leads to this corruption. Initially[0] I also though fetchmail would have the same bug, but not being a fetchmail user myself I wasn't aware that it just hands over mail to the local MTA, in my test case postfix. So it turns out that postfix' local(8) delivery daemon seems to use the mboxo format and is thereby prone to irrecoverable corruption of From_ lines that appear _inside_ mails. (copying from [0]:) What happens in short: Attached are two versions of the same mail, the transcript as I send it via SMTP and what postfix stores in the mbox. You will note that postfix replaces any lines matching (regexp): "^From (.*)$" with: ">From \1" The problem now is, that it does not quote From_ lines already starting with one or more ">" as this is required in the mboxrd format; i.e.: "^>(>*)From (.*)$" would need to be replaced by ">>\1From \2" The effect now is, that mails cannot be unquoted anymore and the corruption added above is obviously irrecoverably. Looking at the local(8) manpage, I haven't found any options that would allow to control the subformat of mbox that is used,... but even if, mboxo would have been a bad default, IMHO. Please: a) fix the issue, by switching to one of the other formats (I guess mboxrd is ideal, as it's backwards compatible, but allows one to fully unquote everything correctly) b) ideally, warn all users via the release-notes/change-log/-announce-mailing-list about this, IMHO, most highly severe corruption. It may be helpful to tell them that any mails stored into mbox via fetchmail, that contained lines matching "^From (.*)$" may be corrupted. Further, the way to find the places of possible corruptions is by matching: "^>From (.*)$" (note the single > which is needed here, as THESE are the cases that may either be quoted From_ lines or just lines that were the text ">From ...". I guess both is not to difficult to be done :) and as said,... when switching to mboxrd (instead of mboxo)... no further problems should occur... a bit more will be quoted, which allows the quoting to be undone correctly. (b) Is IMHO important, to at least notify people that they may have suffered from this over years. Looking around it seems to turn out that most people are not aware that mbox is actually a family of formats and that especially mboxo has its issues. Hope that helps, Chris. [0] https://lists.berlios.de/pipermail/fetchmail-users/2012-October/003258.html
postfix-local.mbox
Description: application/mbox
EHLO a.mail.srv.scientia.net MAIL FROM:<m...@christoph.anton.mitterer.name> RCPT TO:<m...@christoph.anton.mitterer.name> DATA From: m...@christoph.anton.mitterer.name To: m...@christoph.anton.mitterer.name Subject: testmail From From From 0 >From >From >From 1 >>From 2 >>>From 3 from from =46rom =46rom 0 . QUIT
smime.p7s
Description: S/MIME cryptographic signature
