Hi Jason, Thanks for the update, I will be watching KAFKA-2489.
Thanks, -Poorna On Wed, Sep 2, 2015 at 5:31 PM, Jason Gustafson <ja...@confluent.io> wrote: > Hi Poorna, > > We've been looking at the same issue as part of KAFKA-2489. I suspect the > initial rebalance is causing the test to timeout, but we're still > investigating. > > Thanks, > Jason > > On Wed, Sep 2, 2015 at 2:03 PM, Poorna Chandra Tejashvi Reddy < > pctre...@gmail.com> wrote: > > > Hi, > > > > I have checked out the latest code out of > https://github.com/apache/kafka > > based > > on commit id e582447adb4708731aff74aa294e7ce2b30b0a41. Looks like the > > performance test on the new-consumer is broken. > > > > in/kafka-consumer-perf-test.sh --zookeeper zkip:2181 --broker-list > > brokerIp:9092 --topic test --messages 50000 --new-consumer > > > > The test does not return any response. Is this expected and is there a > > better way to test the new-consumer. > > > > > > Thanks, > > > > -Poorna > > > > > > On Thu, Aug 27, 2015 at 2:25 PM, Poorna Chandra Tejashvi Reddy < > > pctre...@gmail.com> wrote: > > > > > Hi, > > > > > > We have built the latest kafka from https://github.com/apache/kafka > > based > > > on this commit id 436b7ddc386eb688ba0f12836710f5e4bcaa06c8 . > > > We ran the performance test on a 3 node kafka cluster. There is a huge > > > throughput degradation using the new-consumer compared to the regular > > > consumer. Below are the numbers that explain the same. > > > > > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > > > brokerIp:9092 --topics test --messages 5000000 : gives a throughput of > > 693 K > > > > > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > > > brokerIp:9092 --topics test --messages 5000000 --new-consumer : gives a > > > throughput of 51k > > > > > > The whole set up is based on ec2, Kafka brokers running on r3.2x large. > > > > > > Are you guys aware of this performance degradation , do you have a JIRA > > > for this, which can be used to track the resolution. > > > > > > > > > Thanks, > > > > > > -Poorna > > > > > >
