What do you mean by "We have a dedicated broker that has no leader
partitions". Are you running anything else on that machine? I think you can
run that tool Guozhang from any machine and don't require it to be a kafka
Broker.

Thanks,

Mayuresh

On Sun, Mar 1, 2015 at 5:44 AM, Gene Robichaux <gene.robich...@match.com>
wrote:

> That is what I am using. The problem is when I run it the CPU spikes on
> the broker I am running it from. I just wanted to know if there was a
> different way.
>
> Gene
>
> Sent from my iPhone
>
> > On Feb 28, 2015, at 10:46 PM, Guozhang Wang <wangg...@gmail.com> wrote:
> >
> > If it is ZK based offset commit, you can use the ConsumerOffsetChecker
> tool
> > in kafka.tools.
> >
> > On Sat, Feb 28, 2015 at 12:32 PM, Gene Robichaux <
> gene.robich...@match.com>
> > wrote:
> >
> >> I think we ZK based offset commit. However I am not certain, I would
> have
> >> to get that from our DEV group. My role is PROD Ops.
> >>
> >> Gene Robichaux
> >> Manager, Database Operations
> >> Match.com
> >> 8300 Douglas Avenue I Suite 800 I Dallas, TX  75225
> >>
> >> -----Original Message-----
> >> From: Jiangjie Qin [mailto:j...@linkedin.com.INVALID]
> >> Sent: Saturday, February 28, 2015 12:06 PM
> >> To: users@kafka.apache.org
> >> Subject: Re: Best way to show lag?
> >>
> >> Are you using Kafka based offset commit or ZK based offset commit?
> >>
> >>> On 2/28/15, 6:16 AM, "Gene Robichaux" <gene.robich...@match.com>
> wrote:
> >>>
> >>> What is the best way to detect consumer lag?
> >>>
> >>> We are running each consumer as a separate group and I am running the
> >>> ConsumerOffsetChecker to assess the partitions and the lag for each
> >>> group/consumer. I run this every 5 minutes. In some cases I run this
> >>> command up to 75 times on each 5 min polling cycle (once for each
> >>> group/consuer). An example of the command is (bin/kafka-run-class.sh
> >>> kafka.tools.ConsumerOffsetChecker --group consumer-group1 --zkconnect
> >>> zkhost:zkport)
> >>>
> >>> The problem I am running into is CPU usage on the broker when these
> >>> commands run. We have a dedicated broker that has no leader partitions,
> >>> but the high CPU still concerns me.
> >>>
> >>> Is there a better way to detect consumer lag? Preferably one that is
> >>> less impactful?
> >>>
> >>>
> >>> Gene Robichaux
> >>> Manager, Database Operations
> >>> Match.com
> >>> 8300 Douglas Avenue I Suite 800 I Dallas, TX  75225
> >
> >
> > --
> > -- Guozhang
>



-- 
-Regards,
Mayuresh R. Gharat
(862) 250-7125

Reply via email to