Hi Jason,
Thanks for the update, I will be watching KAFKA-2489.
Thanks,
-Poorna
On Wed, Sep 2, 2015 at 5:31 PM, Jason Gustafson wrote:
> Hi Poorna,
>
> We've been looking at the same issue as part of KAFKA-2489. I suspect the
> initial rebalance is causing the test to timeout, but we're still
Hi Poorna,
We've been looking at the same issue as part of KAFKA-2489. I suspect the
initial rebalance is causing the test to timeout, but we're still
investigating.
Thanks,
Jason
On Wed, Sep 2, 2015 at 2:03 PM, Poorna Chandra Tejashvi Reddy <
pctre...@gmail.com> wrote:
> Hi,
>
> I have checked
Hi,
I have checked out the latest code out of https://github.com/apache/kafka based
on commit id e582447adb4708731aff74aa294e7ce2b30b0a41. Looks like the
performance test on the new-consumer is broken.
in/kafka-consumer-perf-test.sh --zookeeper zkip:2181 --broker-list
brokerIp:9092 --topic test -
So I’ve been looking at the SSL performance (which potentially also has an
issue unrelated to this). I’d noticed strange behaviour with poll() on the new
consumer but I wasn’t sure whether this was a bug or a feature. On closer
inspection it seems to arise from ConsumerPerformance relying on a c
Tried bisecting, but turns out things were broken for some time. We really
need some system tests in place to avoid letting even new code break for so
long.
At 49026f11781181c38e9d5edb634be9d27245c961 (May 14th), we went from good
performance -> an error due to broker apparently not accepting the
436b7ddc386eb688ba0f12836710f5e4bcaa06c8 is pretty recent and there could
be some current consumer improvement patches that introduces some
regression. I would suggest doing a binary search in the log from
3f8480ccfb011eb43da774737597c597f703e11b
(maybe even earlier?) to do a quick check.
Guozhang
I think this is likely a regression. The two clients had more or less
equivalent performance when we checked in the code (see my post on this
earlier in the year). Looks like maybe we broke something up in the interim?
On my laptop the new consumer perf seems to have dropped from about
~200MB/sec
I don't think the commands are really equivalent despite just adding the
--new-consumer flag. ConsumerPerformance uses a single thread when using
the new consumer (it literally just allocates the consumer, loops until
it's consumed enough, then exits), whereas the old consumer uses a bunch of
addit
Hi,
We have built the latest kafka from https://github.com/apache/kafka based
on this commit id 436b7ddc386eb688ba0f12836710f5e4bcaa06c8 .
We ran the performance test on a 3 node kafka cluster. There is a huge
throughput degradation using the new-consumer compared to the regular
consumer. Below ar
