If it is ZK based offset commit, you can use the ConsumerOffsetChecker tool in kafka.tools.
On Sat, Feb 28, 2015 at 12:32 PM, Gene Robichaux <gene.robich...@match.com> wrote: > I think we ZK based offset commit. However I am not certain, I would have > to get that from our DEV group. My role is PROD Ops. > > Gene Robichaux > Manager, Database Operations > Match.com > 8300 Douglas Avenue I Suite 800 I Dallas, TX 75225 > > -----Original Message----- > From: Jiangjie Qin [mailto:j...@linkedin.com.INVALID] > Sent: Saturday, February 28, 2015 12:06 PM > To: users@kafka.apache.org > Subject: Re: Best way to show lag? > > Are you using Kafka based offset commit or ZK based offset commit? > > On 2/28/15, 6:16 AM, "Gene Robichaux" <gene.robich...@match.com> wrote: > > >What is the best way to detect consumer lag? > > > >We are running each consumer as a separate group and I am running the > >ConsumerOffsetChecker to assess the partitions and the lag for each > >group/consumer. I run this every 5 minutes. In some cases I run this > >command up to 75 times on each 5 min polling cycle (once for each > >group/consuer). An example of the command is (bin/kafka-run-class.sh > >kafka.tools.ConsumerOffsetChecker --group consumer-group1 --zkconnect > >zkhost:zkport) > > > >The problem I am running into is CPU usage on the broker when these > >commands run. We have a dedicated broker that has no leader partitions, > >but the high CPU still concerns me. > > > >Is there a better way to detect consumer lag? Preferably one that is > >less impactful? > > > > > >Gene Robichaux > >Manager, Database Operations > >Match.com > >8300 Douglas Avenue I Suite 800 I Dallas, TX 75225 > > > > -- -- Guozhang
