Hello Otis, actually my operation team told me that they are seeing following JMX MBean on one of the broker.
" kafka.server* >* ReplicaFetcherThread-0-2-host_*<<KafkaHost>>*-port_ *<KafkaPort>*-*<<TopicName>>*-*<<PartitionID>>*-ConsumerLag this is what confusing me.. and when i monitored this Mbean, i saw values coming in negative. Regards, Nitin Kumar Sharma. On Wed, Dec 10, 2014 at 4:34 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Hi, > > On Wed, Dec 10, 2014 at 12:02 AM, nitin sharma < > kumarsharma.ni...@gmail.com> > wrote: > > > thanks for response. But my questions are still not answered > > > > a. Is ConsumerLag is the metrics that is exposed on kafka broker end or > on > > Consumer side > > > > Consumer. > > > > b. Can ConsumerLag value be in negative? > > > > Shouldn't be. Not sure what the meaning of that would be? You see a > negative value somewhere? > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ > > > > > On Tue, Dec 9, 2014 at 5:23 PM, Otis Gospodnetic < > > otis.gospodne...@gmail.com > > > wrote: > > > > > Nitin, > > > > > > If you don't want to DIY there are already tools and solutions for > > > monitoring Kafka, including consumer lag. Ours is mentioned here: > > > > > > > > > http://blog.sematext.com/2013/10/16/announcement-spm-performance-monitoring-for-kafka/ > > > Others have been mentioned on the list a few times before: > > > http://search-hadoop.com/?q=monitor+lag&fc_project=Kafka > > > > > > Otis > > > -- > > > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > > On Tue, Dec 9, 2014 at 4:59 PM, nitin sharma < > > kumarsharma.ni...@gmail.com> > > > wrote: > > > > > > > Hi Neha, Guozhang Wang > > > > > > > > My project requirement is to monitor Consumer Lag Mbean.. > > > > > > > > 1) According to me this MBean is exposed only at the Consumer side > and > > > not > > > > on Broker server. This is because in my initial PoC, i had used > > > > "kafka-consumer-perf-test.sh" and MBean exposed by this consumer had > > > > "ConsumerLag" counter. > > > > > > > > 2) ConsumerLag value can not be in negative. it has to be either zero > > or > > > > positive. is this a correct assumption? i found the following link on > > > > internet where some one had got a negative value : > > > > > > > > > > > > > > > > > > http://mail-archives.apache.org/mod_mbox/kafka-users/201410.mbox/%3cca+ttwkmbjhy_q+io3_7xsqxxfc6hhf7pzap5sfgtzd22stb...@mail.gmail.com%3E > > > > > > > > Regards, > > > > Nitin Kumar Sharma. > > > > > > > > > > > > On Tue, Dec 9, 2014 at 4:47 PM, Guozhang Wang <wangg...@gmail.com> > > > wrote: > > > > > > > > > Nitin, > > > > > > > > > > You have found the place :) Just send an email to users@ mailing > > list > > > is > > > > > fine. > > > > > > > > > > Guozhang > > > > > > > > > > On Tue, Dec 9, 2014 at 1:26 PM, nitin sharma < > > > > kumarsharma.ni...@gmail.com> > > > > > wrote: > > > > > > > > > > > HI, > > > > > > I need some help related to Kafka Consumer Lag monitoring.. > Kindly > > > let > > > > me > > > > > > know how can i post my query to the forum. > > > > > > > > > > > > Regards, > > > > > > Nitin Kumar Sharma. > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > -- Guozhang > > > > > > > > > > > > > > >
