Re: how to identify rogue consumer

2014-10-09 Thread Jun Rao
Yes. Thanks, Jun On Wed, Oct 8, 2014 at 10:53 PM, Steven Wu wrote: > Jun, you mean trace level logging for requestAppender? > log4j.logger.kafka.network.Processor=TRACE, requestAppender > > if it happens again, I can try to enable it. > > On Wed, Oct 8, 2014 at 9:54 PM, Jun Rao wrote: > > > I

Re: how to identify rogue consumer

2014-10-08 Thread Steven Wu
Jun, you mean trace level logging for requestAppender? log4j.logger.kafka.network.Processor=TRACE, requestAppender if it happens again, I can try to enable it. On Wed, Oct 8, 2014 at 9:54 PM, Jun Rao wrote: > If enabled request logging, you can find this out. > > Thanks, > > Jun > > On Wed, Oct

Re: how to identify rogue consumer

2014-10-08 Thread Jun Rao
If enabled request logging, you can find this out. Thanks, Jun On Wed, Oct 8, 2014 at 8:57 PM, Steven Wu wrote: > I have seen very high "Fetch-Consumer-RequestsPerSec" (like 15K) per broker > in a relatively idle cluster. My hypothesis some misbehaving consumer has a > tight polling loop witho

how to identify rogue consumer

2014-10-08 Thread Steven Wu
I have seen very high "Fetch-Consumer-RequestsPerSec" (like 15K) per broker in a relatively idle cluster. My hypothesis some misbehaving consumer has a tight polling loop without any back-off logic with empty fetch. Unfortunately, this metric doesn't have per-topic breakdown like "BytesInPerSec" o