-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 If you can, test the master-HEAD. It contains all coverity fixes I could find. I will not run coverity against 3.1
On 20-07-14 19:06, Reindl Harald wrote: > 77d17ca7724c32dba27eb187d4abfa129ccf6c4e is *way better* but still > not error free, can you please fire coverity again because i think > it's more effective than me building and testing a lot of versions > over hours - smells like one remaining buffer overflow / underrun > > * "Ostrava Tuesday morning" no longer broken * "Panel: E-Mail-Debug > - IMAP-POP3-Download" 4 out of 1000 broken * i have attached the 4 > diff-files * all 4 still always identical broken * two times POP3 * > two times IMAP > > additional info: > > * "Ostrava Tuesday morning" with 23,40 KB *no error* on 100 loops * > "Panel: E-Mail-Debug - IMAP-POP3-Download" is 380 KB large > > so it seems larger messages are more likely to reprocude the > problem, but that's a error rate which let me put > 77d17ca7724c32dba27eb187d4abfa129ccf6c4e in production because > better than current 8a042214ae1d120581740020f4e73c3cf8d3a6c0 > ___________________________________________________ > my stress-test complained within 25 minutes 5 times > > "multipart_message_3", "message_rfc822" and "multipart_message_2" > are from the dbmail-testsuite, "Test 5" is one of my past problem > cases > > Jul 20 18:37:13 POP3 24: EXPECTED: > 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: > 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3) > > Jul 20 18:41:48 POP3 28: EXPECTED: > 60dd43bb886f6c47d89b5dc3ab3f7f764b68eb2e / FOUND: > b99c389d3479d8d407d43045bcd99219e51a8977 (message_rfc822) > > Jul 20 18:45:53 POP3 25: EXPECTED: > 0e2409bcf6921c5da801e1bf4ca4f57be9f07775 / FOUND: > 3ef1aef632e08eef66d77e5a97091797640f3192 (multipart_message_2) > > Jul 20 18:47:00 POP3 01: EXPECTED: > 04073992ce1c1e75d9bd88e0388486dcc80df627 / FOUND: > af6836092d12edbb8be1396ad5af08c804fad2fb (Test 5) > > Jul 20 18:54:47 POP3 01: EXPECTED: > 04073992ce1c1e75d9bd88e0388486dcc80df627 / FOUND: > af6836092d12edbb8be1396ad5af08c804fad2fb (Test 5) > > Am 20.07.2014 17:36, schrieb Reindl Harald (mobile): >> I will give it a try after back at home >> >> if the other reconstruct problem under load still exists i try to >> get back release version to release version until it is away and >> want the commits after upwards >> >> that sort of regressions was solved for sure because we both >> spent a lot of hours last year and finally there where load tests >> over hours without a single mismatch >> >> However, if we really find a guilty commit the interesting >> question will be what assumption leaded to that exactly code and >> how to prevent similar bugs - Autotest are great but can't replay >> real world mixed load, even my stress tests are synthetic at the >> end of the day and can't really simulate 50 different clients at >> the same time acting on completely different messages and what no >> test ever will reproduce is the memory state with possible leaks >> part of the game after several days of running >> >> the race bug on the other hand seems to be worst after restart >> the service - very strange >> >> >> -------- Ursprüngliche Nachricht -------- Von: Paul J Stevens >> <p...@nfg.nl> Gesendet: 20. Juli 2014 17:00:15 MESZ An: DBMail >> mailinglist <dbmail@dbmail.org> Betreff: Re: [Dbmail] 3.1.16 >> regression -> Re: DBMail 3.1.16 released (Re: corrupted >> multi-mime messages) >> >> I can't believe this passed the tests, since this exact message >> is part of the test-suite. Seems the test itself was broken :-( >> >> I just pushed a fix. *Please* test! >> >> On 20-07-14 14:12, Reindl Harald wrote: >>> CAUTION >> >>> 3.1.16 at least has a regression in case of the commit below >>> that's a different problem and the testmessage with downgrading >>> is reconstructed proper >>> http://git.dbmail.eu/paul/dbmail/commit/?h=dbmail_3_1&id=8a042214ae1d120581740020f4e73c3cf8d3a6c0 >> >>> >>> however, the mime-part-id problem is still the same under load >> >>> i sent Paul some debug data offlist because they contain real >>> messages i am allowed by the mailowner to forward only that >>> way, not in public >> >>> Regards Harry >> >>> Am 20.07.2014 13:38, schrieb Reindl Harald: >>>> *attached* a compressed example *diff* broken / non-broken >>>> copy while compare the same mail IMAP / POP3 in a loop >>>> >>>> 33 out of 1000 always identically broken >>>> >>>> it don't matter on which test message i start the loop the >>>> result is the same in case of multi-mime mails >>>> >>>> fact is there are times while running mixed load for a minute >>>> not a single one get broken and in random time windows any >>>> multimime message for some seconds is damaged the same way >>>> _______________________________________________________________ >>>> >>>> >>>> that below is the temp-folder where the loop stores diffs look at >>>> the number at the end, that's the loop-comuter >>>> >>>> at the begin of the loop a horrible result with 19 broken >>>> copies in serie and later 118-125 are again 8 in serie and >>>> they all have the same difference as "example-diff.zip" >>>> >>>> the mixed load now get "multipart_message_3" instead >>>> "multipart_message_6" most of the time broken >>>> _______________________________________________________________ >>>> >>>> >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_002.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_003.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_004.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_005.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_006.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_007.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_009.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_010.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_011.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_012.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_013.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_014.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_015.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_016.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_017.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_018.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_019.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_020.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_021.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_084.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_089.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_118.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_119.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_120.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_121.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_122.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_123.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_124.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_125.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_335.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_364.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_380.diff >>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_707.diff >>>> _______________________________________________ >>>> >>>> Jul 20 13:26:56 POP3 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Jul 20 13:26:56 IMAP 24: EXPECTED: >>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>> 97039dbf12052e527808ca16188971a405fcb529 >>>> (multipart_message_3) Jul 20 13:26:56 IMAP 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Jul 20 13:27:23 POP3 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Jul 20 13:27:23 IMAP 24: EXPECTED: >>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>> 97039dbf12052e527808ca16188971a405fcb529 >>>> (multipart_message_3) Jul 20 13:27:23 IMAP 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Jul 20 13:27:52 POP3 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Jul 20 13:27:52 IMAP 24: EXPECTED: >>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>> 97039dbf12052e527808ca16188971a405fcb529 >>>> (multipart_message_3) Jul 20 13:27:52 IMAP 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Jul 20 13:28:15 POP3 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Jul 20 13:28:15 IMAP 24: EXPECTED: >>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>> 97039dbf12052e527808ca16188971a405fcb529 >>>> (multipart_message_3) Jul 20 13:28:15 IMAP 33: EXPECTED: >>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>> morning) Am 20.07.2014 12:56, schrieb Reindl Harald: >>>>> Hi Paul >>>>> >>>>> sorry but still unconfirmed >>>>> >>>>> see attached screenshot of debug-interface as well as >>>>> "stress-test.txt" which normally should not output >>>>> anything because it prcatically makes a lot of noise while >>>>> check again and aagin the same messages against known >>>>> checksums >>>>> >>>>> the "Ostrava Tuesday morning" and "multipart_message_6" >>>>> which is from the dbmail-autotests imported on my IMAP >>>>> seems to get most likely broken, but others to randomly >>>>> >>>>> "Ostrava Tuesday morning is the one with context to >>>>> "8a042214ae1d120581740020f4e73c3cf8d3a6c0" and was the >>>>> first customer complaint >>>>> >>>>> Regards Harry >>>>> >>>>> Am 20.07.2014 12:30, schrieb Paul J Stevens: >>>>>> It was a buffer overrun which has now been fixed to the >>>>>> best of my abilities. >>>>>> >>>>>> I've done another round of coverity analysis on the >>>>>> master branch, and this doesn't trigger any new warnings >>>>>> with regard to mentioned fix which is also in the >>>>>> dbmail_3_1 branch. >>>>>> >>>>>> I've just tagged and thereby released 3.1.16 which >>>>>> contains just this fix. >>>>>> >>>>>> Being on vacation, I don't have full access to my usual >>>>>> release scripts (home-network is unreachable), so a tag >>>>>> will have to do. >>>>>> >>>>>> But I've also uploaded the tar file seperately to >>>>>> www.dbmail.org, so all the usual download locations >>>>>> should work. >>>>>> >>>>>> On 18-07-14 23:05, Reindl Harald (mobile) wrote: >>>>>>> Interesting - your attachment showed exactly the same >>>>>>> bug I have only randomly for whatever type of messages >>>>>>> and only under load >>>>>>> >>>>>>> I would sell my soul for a predictable reproducer >>>>>>> because in that case debug logging / gdb / strace could >>>>>>> shed some light and since it's always the same want >>>>>>> finally happens it's pretty sure only a few lines of >>>>>>> code in dbmail to fix it >>>>>>> >>>>>>> I am still sure it's exactly the same code path in both >>>>>>> of our cases and only the trigger to went that path is >>>>>>> different >>>>>>> >>>>>>> Maybe Paul has some idea how you can provide debug >>>>>>> info's since you are able to isolate what happens while >>>>>>> my trigger ends in a large mess of mixed debug logs >>>>>>> >>>>>>> >>>>>>> -------- Ursprüngliche Nachricht -------- Von: >>>>>>> Guilherme Souza <gslso...@gmail.com> Gesendet: 18. Juli >>>>>>> 2014 21:20:35 MESZ An: DBMail mailinglist >>>>>>> <dbmail@dbmail.org> Betreff: Re: [Dbmail] corrupted >>>>>>> multi-mime messages >>>>>>> >>>>>>> I was no able to reply to your message. In my case is >>>>>>> not random. >>>>>>> >>>>>>> Every message html formatted with an attached file sent >>>>>>> by ensignia webmail shows the same missing boundaries. >>>>>>> >>>>>>> Every message sent by roundcube or thunderbird shows >>>>>>> correctly. >>>>>>> >>>>>>> >>>>>>> On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile) >>>>>>> <h.rei...@thelounge.net> wrote: >>>>>>>> That is exactly what i reported a few days ago - >>>>>>>> that's for sure a race condition in reconstruction >>>>>>>> because i get predictable identical broken messages >>>>>>>> in a loop while mixed load on my testserver - at the >>>>>>>> same machine 1000 loops receive the same message >>>>>>>> without other clients no broken one > > > > _______________________________________________ DBmail mailing > list DBmail@dbmail.org > http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail > - -- ________________________________________________________________ Paul J Stevens pjstevns @ gmail, twitter, github, linkedin www.nfg.nl/i...@nfg.nl/+31.85.877.99.97 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlPMDs8ACgkQ8iITvBH4zTHQuwCeIktGVQN1UEdB9iZN0+KsZ10O WZQAmgKiyMDSiPgOwc5RsRiIX8SYZnQM =qjVR -----END PGP SIGNATURE----- _______________________________________________ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
