> On Aug. 18, 2014, 5:32 p.m., Joel Koshy wrote: > > I should be able to review this later today. However, as Jun also mentioned > > can you please run the stress test? When I was working on the original > > (WIP) patch it worked but eventually failed (due to various reasons such as > > corrupt message sizes, etc) on a stress test after several segments had > > rolled and after several log cleaner runs - although I didn't get time to > > look into it your patch should have hopefully addressed these issues. > > Manikumar Reddy O wrote: > I tested the patch with my own test code and it is working fine. > > I ran TestLogCleaning stress test. Some times this test is failing. > But i am not getting any broker-side errors/corrupt messages. > > I also ran TestLogCleaning on trunk (without my patch). This test is > failing for multiple topics. > I am looking into TestLogCleaning code and trying fix if any issue. > > I will keep you updated on the testing status.
I successfully ran the TestLogCleaning stress test. I ran the test for 1,5,10 million messages Jun, I removed the usage of MemoryRecords, Compressor.putRecord classes from this patch. Currently Compressor.close() returns a compressed message with offset as number of messages in that compression. (If i compress 10,11,12,13,14,15 message offsets, then the compressed message will have offset 5). Because of this behavior, we can not use this for server-side compression.(For server side, If i compress 10,11,12,13,14,15 message offsets, then the compresed message shoud have offset 15) - Manikumar Reddy ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/24214/#review50901 ----------------------------------------------------------- On Oct. 3, 2014, 1:50 p.m., Manikumar Reddy O wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/24214/ > ----------------------------------------------------------- > > (Updated Oct. 3, 2014, 1:50 p.m.) > > > Review request for kafka. > > > Bugs: KAFKA-1374 > https://issues.apache.org/jira/browse/KAFKA-1374 > > > Repository: kafka > > > Description > ------- > > fixed couple of bugs and updating stress test details > > > Diffs > ----- > > core/src/main/scala/kafka/log/LogCleaner.scala > c20de4ad4734c0bd83c5954fdb29464a27b91dff > core/src/main/scala/kafka/tools/TestLogCleaning.scala > 1d4ea93f2ba8d4d4d47a307cd47f54a15d3d30dd > core/src/test/scala/unit/kafka/log/LogCleanerIntegrationTest.scala > 5bfa764638e92f217d0ff7108ec8f53193c22978 > > Diff: https://reviews.apache.org/r/24214/diff/ > > > Testing > ------- > > > Thanks, > > Manikumar Reddy O > >
