Hi, We have built the latest kafka from https://github.com/apache/kafka based on this commit id 436b7ddc386eb688ba0f12836710f5e4bcaa06c8 . We ran the performance test on a 3 node kafka cluster. There is a huge throughput degradation using the new-consumer compared to the regular consumer. Below are the numbers that explain the same.
bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list brokerIp:9092 --topics test --messages 5000000 : gives a throughput of 693 K bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list brokerIp:9092 --topics test --messages 5000000 --new-consumer : gives a throughput of 51k The whole set up is based on ec2, Kafka brokers running on r3.2x large. Are you guys aware of this performance degradation , do you have a JIRA for this, which can be used to track the resolution. Thanks, -Poorna
