Hi Guozhang, The throughput refers to the broker side throughput. The peak time throughput of broker for the topic is around (9-10 MBps) but with slow consumers the throughput went down to 5-6 MBps.
I consume data using a consumer group. Consumer group needs 5 consumers to consume the entire data. To simulate slow consumer, i shut down 4 of the consumers and only one consumer is active. On Tue, Mar 18, 2014 at 4:49 PM, Guozhang Wang <wangg...@gmail.com> wrote: > Hello Abhinav, > > When you say "slow consumer set on topic1", do you mean you have a consumer > group consuming from topic1, with one of them slow, or do you mean all > consumers are slow in that group, or do you use a simple consumer? When you > say "throughput", do you mean consumer side throughput, or broker side > throughput? > > Guozhang > > > On Tue, Mar 18, 2014 at 4:25 PM, Abhinav Anand <ab.rv...@gmail.com> wrote: > > > I was testing the impact of slow consumers on Kafka. I had slow consumers > > set on topic1. The throughput of topic1 went down by 40-50% for the peak > > period (~ 9-10 MBps). Though the throughput for other topics was not > > affected at all. > > > > I couldn't find any error logs on the server side or producer. The Kafka > > brokers are running on VMs with 16GB RAM and 4 Cores. > > > > The consumer configuration is pretty basic and only has zookeeper > > information, timeout details. > > > > Have you guys tested kafka behaviour for slow consumers? Any broker level > > configuration for better performance ? Any insights on how do handle such > > rogue consumers as a part of operation ? > > > > -- > > Abhinav Anand > > > > > > -- > -- Guozhang > -- Abhinav Anand
