Re: consumer offset tool and JMX metrics do not match

That thread is describing my exact experience. The last reply is in June. Has anything changed? Are we supposed to stop trusting JMX and try to use borrow or integrate with Consumer Offset Checker? Shouldn't the documentation reflect this matter as I am sure everyone will try to use JMX first since

Re: consumer offset tool and JMX metrics do not match

Hi Allen, I was referring to one of the issues here: http://search-hadoop.com/m/uyzND1XVyK12UNtd32/kafka+orphaned/v=threaded This linked thread discusses one of such issues where consumer lag was not reported correctly. Regards, Prabhjot On Sun, Nov 15, 2015 at 7:04 AM, allen chan wrote: > I b

Re: consumer offset tool and JMX metrics do not match

sets >>> >>> >>> > When the Lag value in the Kafka consumer JMX is high (for example 5M), >>> > ConsumerOffsetChecker shows a matching number. >>> > >>> > I am running kafka_2.10-0.8.2.1 >>> > >>> > Osama >

Re: consumer offset tool and JMX metrics do not match

a_2.10-0.8.2.1 >> > >> > Osama >> > >> > -----Original Message----- >> > From: allen chan [mailto:allen.michael.c...@gmail.com] >> > Sent: Saturday, November 14, 2015 8:34 PM >> > To: users@kafka.apache.org >> > Subject: Re

Re: consumer offset tool and JMX metrics do not match

gt; > From: allen chan [mailto:allen.michael.c...@gmail.com] > > Sent: Saturday, November 14, 2015 8:34 PM > > To: users@kafka.apache.org > > Subject: Re: consumer offset tool and JMX metrics do not match > > > > I believe producers / brokers / and consumers has bee

Re: consumer offset tool and JMX metrics do not match

t; -Original Message- > From: allen chan [mailto:allen.michael.c...@gmail.com] > Sent: Saturday, November 14, 2015 8:34 PM > To: users@kafka.apache.org > Subject: Re: consumer offset tool and JMX metrics do not match > > I believe producers / brokers / and consumers has bee

RE: consumer offset tool and JMX metrics do not match

matching number. I am running kafka_2.10-0.8.2.1 Osama -Original Message- From: allen chan [mailto:allen.michael.c...@gmail.com] Sent: Saturday, November 14, 2015 8:34 PM To: users@kafka.apache.org Subject: Re: consumer offset tool and JMX metrics do not match I believe producers / brokers

Re: consumer offset tool and JMX metrics do not match

I believe producers / brokers / and consumers has been restarted at different times. What do you think the issue is? On Sat, Nov 14, 2015 at 2:52 AM, Prabhjot Bharaj wrote: > Hi, > > Did any of your brokers/consumers undergo a restart between the time after > you had started the consumption and

Re: consumer offset tool and JMX metrics do not match

Hi, Did any of your brokers/consumers undergo a restart between the time after you had started the consumption and until you see this issue ? Thanks, Prabhjot On Sat, Nov 14, 2015 at 5:53 AM, allen chan wrote: > I also looked at this metric in JMX and it is also 0 > *kafka.consumer:type=Cons

Re: consumer offset tool and JMX metrics do not match

I also looked at this metric in JMX and it is also 0 *kafka.consumer:type=ConsumerFetcherManager,name=MaxLag,clientId=logstash* On Fri, Nov 13, 2015 at 4:06 PM, allen chan wrote: > Hi All, > > I am comparing the output from kafka.tools.ConsumerOffsetChecker vs JMX > (kafka.server:type=FetcherLag

consumer offset tool and JMX metrics do not match

Hi All, I am comparing the output from kafka.tools.ConsumerOffsetChecker vs JMX (kafka.server:type=FetcherLagMetrics,name=ConsumerLag,clientId=logstash,topic=logstash_fdm,partition=*) and they do not match. ConsumerOffsetChecker is showing ~60 Lag per partition and JMX shows 0 for all partitions.