436b7ddc386eb688ba0f12836710f5e4bcaa06c8 is pretty recent and there could be some current consumer improvement patches that introduces some regression. I would suggest doing a binary search in the log from 3f8480ccfb011eb43da774737597c597f703e11b (maybe even earlier?) to do a quick check.
Guozhang On Thu, Aug 27, 2015 at 4:39 PM, Jay Kreps <j...@confluent.io> wrote: > I think this is likely a regression. The two clients had more or less > equivalent performance when we checked in the code (see my post on this > earlier in the year). Looks like maybe we broke something up in the > interim? > > On my laptop the new consumer perf seems to have dropped from about > ~200MB/sec to about 2MB/sec. > > -Jay > > > On Thu, Aug 27, 2015 at 4:21 PM, Ewen Cheslack-Postava <e...@confluent.io> > wrote: > > > I don't think the commands are really equivalent despite just adding the > > --new-consumer flag. ConsumerPerformance uses a single thread when using > > the new consumer (it literally just allocates the consumer, loops until > > it's consumed enough, then exits), whereas the old consumer uses a bunch > of > > additional threads. > > > > To really compare performance, someone would have to think through a fair > > way to compare them -- the two operate so differently that you'd have to > be > > very careful to get an apples-to-apples comparison. > > > > By the way, membership in consumer groups should be a lot cheaper with > the > > new consumer (the ZK coordination issues with lots of consumers aren't a > > problem since ZK is not involved), so you can probably scale up the > number > > of consumer threads with little impact. It might be nice to patch the > > consumer perf test to respect the # of threads setting, which might be a > > first step to getting a more reasonable comparison. > > > > -Ewen > > > > On Thu, Aug 27, 2015 at 11:25 AM, Poorna Chandra Tejashvi Reddy < > > pctre...@gmail.com> wrote: > > > > > Hi, > > > > > > We have built the latest kafka from https://github.com/apache/kafka > > based > > > on this commit id 436b7ddc386eb688ba0f12836710f5e4bcaa06c8 . > > > We ran the performance test on a 3 node kafka cluster. There is a huge > > > throughput degradation using the new-consumer compared to the regular > > > consumer. Below are the numbers that explain the same. > > > > > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > > > brokerIp:9092 --topics test --messages 5000000 : gives a throughput of > > 693 > > > K > > > > > > bin/kafka-consumer-perf-test.sh --zookeeper zkIp:2181 --broker-list > > > brokerIp:9092 --topics test --messages 5000000 --new-consumer : gives a > > > throughput of 51k > > > > > > The whole set up is based on ec2, Kafka brokers running on r3.2x large. > > > > > > Are you guys aware of this performance degradation , do you have a JIRA > > for > > > this, which can be used to track the resolution. > > > > > > > > > Thanks, > > > > > > -Poorna > > > > > > > > > > > -- > > Thanks, > > Ewen > > > -- -- Guozhang
