-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I can't believe this passed the tests, since this exact message is part of the test-suite. Seems the test itself was broken :-(
I just pushed a fix. *Please* test! On 20-07-14 14:12, Reindl Harald wrote: > CAUTION > > 3.1.16 at least has a regression in case of the commit below that's > a different problem and the testmessage with downgrading is > reconstructed proper > http://git.dbmail.eu/paul/dbmail/commit/?h=dbmail_3_1&id=8a042214ae1d120581740020f4e73c3cf8d3a6c0 > > however, the mime-part-id problem is still the same under load > > i sent Paul some debug data offlist because they contain real > messages i am allowed by the mailowner to forward only that way, > not in public > > Regards Harry > > Am 20.07.2014 13:38, schrieb Reindl Harald: >> *attached* a compressed example *diff* broken / non-broken copy >> while compare the same mail IMAP / POP3 in a loop >> >> 33 out of 1000 always identically broken >> >> it don't matter on which test message i start the loop the result >> is the same in case of multi-mime mails >> >> fact is there are times while running mixed load for a minute not >> a single one get broken and in random time windows any multimime >> message for some seconds is damaged the same way >> _______________________________________________________________ >> >> that below is the temp-folder where the loop stores diffs look at >> the number at the end, that's the loop-comuter >> >> at the begin of the loop a horrible result with 19 broken copies >> in serie and later 118-125 are again 8 in serie and they all have >> the same difference as "example-diff.zip" >> >> the mixed load now get "multipart_message_3" instead >> "multipart_message_6" most of the time broken >> _______________________________________________________________ >> >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_002.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_003.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_004.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_005.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_006.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_007.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_009.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_010.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_011.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_012.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_013.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_014.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_015.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_016.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_017.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_018.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_019.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_020.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_021.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_084.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_089.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_118.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_119.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_120.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_121.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_122.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_123.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_124.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_125.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_335.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_364.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_380.diff >> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_707.diff >> _______________________________________________ >> >> Jul 20 13:26:56 POP3 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Jul 20 13:26:56 IMAP 24: EXPECTED: >> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3) >> Jul 20 13:26:56 IMAP 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Jul 20 13:27:23 POP3 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Jul 20 13:27:23 IMAP 24: EXPECTED: >> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3) >> Jul 20 13:27:23 IMAP 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Jul 20 13:27:52 POP3 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Jul 20 13:27:52 IMAP 24: EXPECTED: >> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3) >> Jul 20 13:27:52 IMAP 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Jul 20 13:28:15 POP3 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Jul 20 13:28:15 IMAP 24: EXPECTED: >> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3) >> Jul 20 13:28:15 IMAP 33: EXPECTED: >> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >> morning) Am 20.07.2014 12:56, schrieb Reindl Harald: >>> Hi Paul >>> >>> sorry but still unconfirmed >>> >>> see attached screenshot of debug-interface as well as >>> "stress-test.txt" which normally should not output anything >>> because it prcatically makes a lot of noise while check again >>> and aagin the same messages against known checksums >>> >>> the "Ostrava Tuesday morning" and "multipart_message_6" which >>> is from the dbmail-autotests imported on my IMAP seems to get >>> most likely broken, but others to randomly >>> >>> "Ostrava Tuesday morning is the one with context to >>> "8a042214ae1d120581740020f4e73c3cf8d3a6c0" and was the first >>> customer complaint >>> >>> Regards Harry >>> >>> Am 20.07.2014 12:30, schrieb Paul J Stevens: >>>> It was a buffer overrun which has now been fixed to the best >>>> of my abilities. >>>> >>>> I've done another round of coverity analysis on the master >>>> branch, and this doesn't trigger any new warnings with regard >>>> to mentioned fix which is also in the dbmail_3_1 branch. >>>> >>>> I've just tagged and thereby released 3.1.16 which contains >>>> just this fix. >>>> >>>> Being on vacation, I don't have full access to my usual >>>> release scripts (home-network is unreachable), so a tag will >>>> have to do. >>>> >>>> But I've also uploaded the tar file seperately to >>>> www.dbmail.org, so all the usual download locations should >>>> work. >>>> >>>> On 18-07-14 23:05, Reindl Harald (mobile) wrote: >>>>> Interesting - your attachment showed exactly the same bug I >>>>> have only randomly for whatever type of messages and only >>>>> under load >>>>> >>>>> I would sell my soul for a predictable reproducer because >>>>> in that case debug logging / gdb / strace could shed some >>>>> light and since it's always the same want finally happens >>>>> it's pretty sure only a few lines of code in dbmail to fix >>>>> it >>>>> >>>>> I am still sure it's exactly the same code path in both of >>>>> our cases and only the trigger to went that path is >>>>> different >>>>> >>>>> Maybe Paul has some idea how you can provide debug info's >>>>> since you are able to isolate what happens while my trigger >>>>> ends in a large mess of mixed debug logs >>>>> >>>>> >>>>> -------- Ursprüngliche Nachricht -------- Von: Guilherme >>>>> Souza <gslso...@gmail.com> Gesendet: 18. Juli 2014 21:20:35 >>>>> MESZ An: DBMail mailinglist <dbmail@dbmail.org> Betreff: >>>>> Re: [Dbmail] corrupted multi-mime messages >>>>> >>>>> I was no able to reply to your message. In my case is not >>>>> random. >>>>> >>>>> Every message html formatted with an attached file sent by >>>>> ensignia webmail shows the same missing boundaries. >>>>> >>>>> Every message sent by roundcube or thunderbird shows >>>>> correctly. >>>>> >>>>> >>>>> On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile) >>>>> <h.rei...@thelounge.net> wrote: >>>>>> That is exactly what i reported a few days ago - that's >>>>>> for sure a race condition in reconstruction because i get >>>>>> predictable identical broken messages in a loop while >>>>>> mixed load on my testserver - at the same machine 1000 >>>>>> loops receive the same message without other clients no >>>>>> broken one > > > > > _______________________________________________ DBmail mailing > list DBmail@dbmail.org > http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail > - -- ________________________________________________________________ Paul J Stevens pjstevns @ gmail, twitter, github, linkedin www.nfg.nl/i...@nfg.nl/+31.85.877.99.97 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlPL2X8ACgkQ8iITvBH4zTGatACePuAvKXmihNJxvLbq2l66If3j XV4An0CldZCEP3L29ubcrUry6A4K1ru2 =0rVS -----END PGP SIGNATURE----- _______________________________________________ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
