another complaint 5 minutes ago "i received last week some empty messages, complained by the sender and they are saying it's in the responsibility of the receiver"
don't get me wrong but fixes like below in that context let me despair :-( - strncat(boundary, s, buflen); + strncat(boundary, s, min(i, buflen)); Am 21.07.2014 13:01, schrieb Reindl Harald: > i am out of ideas, customer complaint that two messages > with a pdf invoice are broken and i really don't know > that to answer - i can't tell a customer that get > incoming mails undamaged is some sort of luck > > that's exactly the reconstruct problem "--" without mime-id > > Date: Mon, 21 Jul 2014 10:43:17 +0200 > Date: Mon, 21 Jul 2014 10:43:33 +0200 > _______________________________________________________ > > -- > Content-Type: text/html; > > -- > Content-Type: application/pdf; > _______________________________________________________ > > Am 20.07.2014 20:56, schrieb Reindl Harald: >> Am 20.07.2014 20:47, schrieb Paul J Stevens: >>> If you can, test the master-HEAD. It contains all coverity fixes I >>> could find. I will not run coverity against 3.1 >> >> the problem is that i can't push MASTER to production and the >> problems are affecting dbmail 3.1 stable >> >> i did not mean fix anything coverity lists but maybe >> with the diffs what happening to messages coverity >> can give a educated hint >> >> since when it happens it's always the same result >> having lines with "--" instead "-- mimeid" it is >> likely a single specific bug :-( >> >>> On 20-07-14 19:06, Reindl Harald wrote: >>>> 77d17ca7724c32dba27eb187d4abfa129ccf6c4e is *way better* but still >>>> not error free, can you please fire coverity again because i think >>>> it's more effective than me building and testing a lot of versions >>>> over hours - smells like one remaining buffer overflow / underrun >>> >>>> * "Ostrava Tuesday morning" no longer broken * "Panel: E-Mail-Debug >>>> - IMAP-POP3-Download" 4 out of 1000 broken * i have attached the 4 >>>> diff-files * all 4 still always identical broken * two times POP3 * >>>> two times IMAP >>> >>>> additional info: >>> >>>> * "Ostrava Tuesday morning" with 23,40 KB *no error* on 100 loops * >>>> "Panel: E-Mail-Debug - IMAP-POP3-Download" is 380 KB large >>> >>>> so it seems larger messages are more likely to reprocude the >>>> problem, but that's a error rate which let me put >>>> 77d17ca7724c32dba27eb187d4abfa129ccf6c4e in production because >>>> better than current 8a042214ae1d120581740020f4e73c3cf8d3a6c0 >>>> ___________________________________________________ >>> >>>> my stress-test complained within 25 minutes 5 times >>> >>>> "multipart_message_3", "message_rfc822" and "multipart_message_2" >>>> are from the dbmail-testsuite, "Test 5" is one of my past problem >>>> cases >>> >>>> Jul 20 18:37:13 POP3 24: EXPECTED: >>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>> 97039dbf12052e527808ca16188971a405fcb529 (multipart_message_3) >>> >>>> Jul 20 18:41:48 POP3 28: EXPECTED: >>>> 60dd43bb886f6c47d89b5dc3ab3f7f764b68eb2e / FOUND: >>>> b99c389d3479d8d407d43045bcd99219e51a8977 (message_rfc822) >>> >>>> Jul 20 18:45:53 POP3 25: EXPECTED: >>>> 0e2409bcf6921c5da801e1bf4ca4f57be9f07775 / FOUND: >>>> 3ef1aef632e08eef66d77e5a97091797640f3192 (multipart_message_2) >>> >>>> Jul 20 18:47:00 POP3 01: EXPECTED: >>>> 04073992ce1c1e75d9bd88e0388486dcc80df627 / FOUND: >>>> af6836092d12edbb8be1396ad5af08c804fad2fb (Test 5) >>> >>>> Jul 20 18:54:47 POP3 01: EXPECTED: >>>> 04073992ce1c1e75d9bd88e0388486dcc80df627 / FOUND: >>>> af6836092d12edbb8be1396ad5af08c804fad2fb (Test 5) >>> >>>> Am 20.07.2014 17:36, schrieb Reindl Harald (mobile): >>>>> I will give it a try after back at home >>>>> >>>>> if the other reconstruct problem under load still exists i try to >>>>> get back release version to release version until it is away and >>>>> want the commits after upwards >>>>> >>>>> that sort of regressions was solved for sure because we both >>>>> spent a lot of hours last year and finally there where load tests >>>>> over hours without a single mismatch >>>>> >>>>> However, if we really find a guilty commit the interesting >>>>> question will be what assumption leaded to that exactly code and >>>>> how to prevent similar bugs - Autotest are great but can't replay >>>>> real world mixed load, even my stress tests are synthetic at the >>>>> end of the day and can't really simulate 50 different clients at >>>>> the same time acting on completely different messages and what no >>>>> test ever will reproduce is the memory state with possible leaks >>>>> part of the game after several days of running >>>>> >>>>> the race bug on the other hand seems to be worst after restart >>>>> the service - very strange >>>>> >>>>> >>>>> -------- Ursprüngliche Nachricht -------- Von: Paul J Stevens >>>>> <p...@nfg.nl> Gesendet: 20. Juli 2014 17:00:15 MESZ An: DBMail >>>>> mailinglist <dbmail@dbmail.org> Betreff: Re: [Dbmail] 3.1.16 >>>>> regression -> Re: DBMail 3.1.16 released (Re: corrupted >>>>> multi-mime messages) >>>>> >>>>> I can't believe this passed the tests, since this exact message >>>>> is part of the test-suite. Seems the test itself was broken :-( >>>>> >>>>> I just pushed a fix. *Please* test! >>>>> >>>>> On 20-07-14 14:12, Reindl Harald wrote: >>>>>> CAUTION >>>>> >>>>>> 3.1.16 at least has a regression in case of the commit below >>>>>> that's a different problem and the testmessage with downgrading >>>>>> is reconstructed proper >>>>>> http://git.dbmail.eu/paul/dbmail/commit/?h=dbmail_3_1&id=8a042214ae1d120581740020f4e73c3cf8d3a6c0 >>>>> >>>>>> >>>>>> >>> however, the mime-part-id problem is still the same under load >>>>> >>>>>> i sent Paul some debug data offlist because they contain real >>>>>> messages i am allowed by the mailowner to forward only that >>>>>> way, not in public >>>>> >>>>>> Regards Harry >>>>> >>>>>> Am 20.07.2014 13:38, schrieb Reindl Harald: >>>>>>> *attached* a compressed example *diff* broken / non-broken >>>>>>> copy while compare the same mail IMAP / POP3 in a loop >>>>>>> >>>>>>> 33 out of 1000 always identically broken >>>>>>> >>>>>>> it don't matter on which test message i start the loop the >>>>>>> result is the same in case of multi-mime mails >>>>>>> >>>>>>> fact is there are times while running mixed load for a minute >>>>>>> not a single one get broken and in random time windows any >>>>>>> multimime message for some seconds is damaged the same way >>>>>>> _______________________________________________________________ >>>>>>> >>>>>>> >>>>>>> >>> that below is the temp-folder where the loop stores diffs look at >>>>>>> the number at the end, that's the loop-comuter >>>>>>> >>>>>>> at the begin of the loop a horrible result with 19 broken >>>>>>> copies in serie and later 118-125 are again 8 in serie and >>>>>>> they all have the same difference as "example-diff.zip" >>>>>>> >>>>>>> the mixed load now get "multipart_message_3" instead >>>>>>> "multipart_message_6" most of the time broken >>>>>>> _______________________________________________________________ >>>>>>> >>>>>>> >>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_002.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_003.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_004.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_005.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_006.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_007.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_009.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_010.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_011.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_012.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_013.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_014.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_015.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_016.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_017.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_018.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_019.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_020.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_021.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_084.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_089.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_118.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_119.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_120.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_121.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_122.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_123.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_124.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_125.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_335.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_364.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_380.diff >>>>>>> dbmail_c6442c520ba8b8a45f1751eb666caeec9ea1ed40_6_707.diff >>>>>>> _______________________________________________ >>>>>>> >>>>>>> Jul 20 13:26:56 POP3 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Jul 20 13:26:56 IMAP 24: EXPECTED: >>>>>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>>>>> 97039dbf12052e527808ca16188971a405fcb529 >>>>>>> (multipart_message_3) Jul 20 13:26:56 IMAP 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Jul 20 13:27:23 POP3 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Jul 20 13:27:23 IMAP 24: EXPECTED: >>>>>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>>>>> 97039dbf12052e527808ca16188971a405fcb529 >>>>>>> (multipart_message_3) Jul 20 13:27:23 IMAP 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Jul 20 13:27:52 POP3 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Jul 20 13:27:52 IMAP 24: EXPECTED: >>>>>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>>>>> 97039dbf12052e527808ca16188971a405fcb529 >>>>>>> (multipart_message_3) Jul 20 13:27:52 IMAP 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Jul 20 13:28:15 POP3 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Jul 20 13:28:15 IMAP 24: EXPECTED: >>>>>>> 2210e351665b9771a6eefab2cb0c8bf43a04fcea / FOUND: >>>>>>> 97039dbf12052e527808ca16188971a405fcb529 >>>>>>> (multipart_message_3) Jul 20 13:28:15 IMAP 33: EXPECTED: >>>>>>> 16292abceedfc6f37707cb1adf35e1e1a46d7d2b / FOUND: >>>>>>> bd17702e6872ebdd60c38f7ef439f9f30a5b9721 (Ostrava Tuesday >>>>>>> morning) Am 20.07.2014 12:56, schrieb Reindl Harald: >>>>>>>> Hi Paul >>>>>>>> >>>>>>>> sorry but still unconfirmed >>>>>>>> >>>>>>>> see attached screenshot of debug-interface as well as >>>>>>>> "stress-test.txt" which normally should not output >>>>>>>> anything because it prcatically makes a lot of noise while >>>>>>>> check again and aagin the same messages against known >>>>>>>> checksums >>>>>>>> >>>>>>>> the "Ostrava Tuesday morning" and "multipart_message_6" >>>>>>>> which is from the dbmail-autotests imported on my IMAP >>>>>>>> seems to get most likely broken, but others to randomly >>>>>>>> >>>>>>>> "Ostrava Tuesday morning is the one with context to >>>>>>>> "8a042214ae1d120581740020f4e73c3cf8d3a6c0" and was the >>>>>>>> first customer complaint >>>>>>>> >>>>>>>> Regards Harry >>>>>>>> >>>>>>>> Am 20.07.2014 12:30, schrieb Paul J Stevens: >>>>>>>>> It was a buffer overrun which has now been fixed to the >>>>>>>>> best of my abilities. >>>>>>>>> >>>>>>>>> I've done another round of coverity analysis on the >>>>>>>>> master branch, and this doesn't trigger any new warnings >>>>>>>>> with regard to mentioned fix which is also in the >>>>>>>>> dbmail_3_1 branch. >>>>>>>>> >>>>>>>>> I've just tagged and thereby released 3.1.16 which >>>>>>>>> contains just this fix. >>>>>>>>> >>>>>>>>> Being on vacation, I don't have full access to my usual >>>>>>>>> release scripts (home-network is unreachable), so a tag >>>>>>>>> will have to do. >>>>>>>>> >>>>>>>>> But I've also uploaded the tar file seperately to >>>>>>>>> www.dbmail.org, so all the usual download locations >>>>>>>>> should work. >>>>>>>>> >>>>>>>>> On 18-07-14 23:05, Reindl Harald (mobile) wrote: >>>>>>>>>> Interesting - your attachment showed exactly the same >>>>>>>>>> bug I have only randomly for whatever type of messages >>>>>>>>>> and only under load >>>>>>>>>> >>>>>>>>>> I would sell my soul for a predictable reproducer >>>>>>>>>> because in that case debug logging / gdb / strace could >>>>>>>>>> shed some light and since it's always the same want >>>>>>>>>> finally happens it's pretty sure only a few lines of >>>>>>>>>> code in dbmail to fix it >>>>>>>>>> >>>>>>>>>> I am still sure it's exactly the same code path in both >>>>>>>>>> of our cases and only the trigger to went that path is >>>>>>>>>> different >>>>>>>>>> >>>>>>>>>> Maybe Paul has some idea how you can provide debug >>>>>>>>>> info's since you are able to isolate what happens while >>>>>>>>>> my trigger ends in a large mess of mixed debug logs >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -------- Ursprüngliche Nachricht -------- Von: >>>>>>>>>> Guilherme Souza <gslso...@gmail.com> Gesendet: 18. Juli >>>>>>>>>> 2014 21:20:35 MESZ An: DBMail mailinglist >>>>>>>>>> <dbmail@dbmail.org> Betreff: Re: [Dbmail] corrupted >>>>>>>>>> multi-mime messages >>>>>>>>>> >>>>>>>>>> I was no able to reply to your message. In my case is >>>>>>>>>> not random. >>>>>>>>>> >>>>>>>>>> Every message html formatted with an attached file sent >>>>>>>>>> by ensignia webmail shows the same missing boundaries. >>>>>>>>>> >>>>>>>>>> Every message sent by roundcube or thunderbird shows >>>>>>>>>> correctly. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, Jul 18, 2014 at 2:31 PM, Reindl Harald (mobile) >>>>>>>>>> <h.rei...@thelounge.net> wrote: >>>>>>>>>>> That is exactly what i reported a few days ago - >>>>>>>>>>> that's for sure a race condition in reconstruction >>>>>>>>>>> because i get predictable identical broken messages >>>>>>>>>>> in a loop while mixed load on my testserver - at the >>>>>>>>>>> same machine 1000 loops receive the same message >>>>>>>>>>> without other clients no broken one
signature.asc
Description: OpenPGP digital signature
_______________________________________________ DBmail mailing list DBmail@dbmail.org http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail
