In general this is expected. Consumers generally read from the OS cache and so they do no I/O. However a slow consumer may fall out of this cached portion of the log and do real reads. These reads will compete with writes for disk bandwidth.
However what is puzzling is that you mention that other topics (presumably on the same machine?) are not effected. This sort of shoots down that theory... -Jay On Tue, Mar 18, 2014 at 4:25 PM, Abhinav Anand <ab.rv...@gmail.com> wrote: > I was testing the impact of slow consumers on Kafka. I had slow consumers > set on topic1. The throughput of topic1 went down by 40-50% for the peak > period (~ 9-10 MBps). Though the throughput for other topics was not > affected at all. > > I couldn't find any error logs on the server side or producer. The Kafka > brokers are running on VMs with 16GB RAM and 4 Cores. > > The consumer configuration is pretty basic and only has zookeeper > information, timeout details. > > Have you guys tested kafka behaviour for slow consumers? Any broker level > configuration for better performance ? Any insights on how do handle such > rogue consumers as a part of operation ? > > -- > Abhinav Anand >
