Re: Perf producer/consumers for compacted topics

2016-05-18 Thread Tom Crayford
Hi, I'm interested in benchmarking the impact of compaction on producers and consumers and long term cluster stability. That's not *quite* the impact of it on the server side, but it certainly plays into it. For example, I'd like to be able to answer "in configuration X, if we write N messages int

Re: Perf producer/consumers for compacted topics

2016-05-18 Thread Manikumar Reddy
Hi, There is a kafka.tools.TestLogCleaning tool, which is used to stress test the compaction feature. This tool validates the correctness of compaction process. This tool can be improved for perf testing. I think you want to benchmark server side compaction process. Currently we have few compact