Re: Benchmarking kafka performance

2016-09-23 Thread Kaufman Ng
Kafka includes producer and consumer performance test tools: - kafka-producer-perf-test - kafka-consumer-perf-test You can find out about more background on the tools here (note that the details are dated): https://engineering.linkedin.com/kafka/benchmarking-apache-kafka-2-million-writes-second-

Re: Benchmarking kafka performance

2016-09-23 Thread Kenny Gorman
Vadim, We mostly made this little script as a joke. Remember the unix utility ‘yes’? It does in fact work if you want to simply direct some random load at Kafka to test things. Throw it into Docker and run a bunch of them. ;-) https://github.com/Eventador/evtools/tree/master/yesbench In terms

Re: Benchmarking kafka performance

2016-09-22 Thread Dominik Safaric
> Good morning. Which benchmarking tools we should use to compare performance > of 0.8 and 0.10 versions? Which metrics should we monitor ? It dependents on your use-case/requirements (if any). I suggest you take a look at a general high-level benchmark made by LinkedIn. They’ve focused onto

Re: Benchmarking kafka performance

2016-09-22 Thread Vadim Keylis
Good afternoon. Any suggestions regarding benchmark tool would be greatly appreciated. Thanks On Mon, Sep 19, 2016 at 8:18 AM, Vadim Keylis wrote: > Good morning. Which benchmarking tools we should use to compare > performance of 0.8 and 0.10 versions? Which metrics should we monitor ? > > Tha