Re: Kafka Queue Depth Metrics

2014-10-05 Thread Jun Rao
This is a jmx on the consumer side. The server side metric is for the internal replication consumer and is only available if you have a replication factor more than 1. Thanks, Jun On Fri, Oct 3, 2014 at 4:28 AM, Shah, Devang1 wrote: > I tried hooking up jconsole to Kafka server but could not s

Re: Kafka Queue Depth Metrics

2014-10-03 Thread Shah, Devang1
I tried hooking up jconsole to Kafka server but could not see the "kafka.consumer" MBean. I am using kafka_2.9.2-0.8.1.1 Am I going wrong anywhere ? Thanks, Devang

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Arnaud Lawson
Ok thank you. Arnaud- On Tue, Apr 1, 2014 at 10:50 AM, Jun Rao wrote: > Search for MaxLag in http://kafka.apache.org/documentation.html#monitoring > . > Note that this is only available in 0.8.0 and above. > > Thanks, > > Jun > > > On Tue, Apr 1, 2014 at 7:21 AM, Arnaud Lawson >wrote: > > > T

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Jun Rao
Search for MaxLag in http://kafka.apache.org/documentation.html#monitoring. Note that this is only available in 0.8.0 and above. Thanks, Jun On Tue, Apr 1, 2014 at 7:21 AM, Arnaud Lawson wrote: > Thanks Jun, > > So on the consumer side is the metric (measuring the # of messages a > consumer is

Re: Kafka Queue Depth Metrics

2014-04-01 Thread Arnaud Lawson
Thanks Jun, So on the consumer side is the metric (measuring the # of messages a consumer is behind) called getOffsetLag? Reference: https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring Arnaud- On Tue, Apr 1, 2014 at 12:16 AM, Jun Rao wrote: > We also have a me

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Jun Rao
We also have a metric in the consumer that measures the # of messages a consumer is behind. Thanks, Jun On Mon, Mar 31, 2014 at 8:36 AM, Arnaud Lawson wrote: > Hi, > > Does anyone know what the JMX metric(s) are for determining the depth of > the kafka queue? I basically want to know the amoun

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Guozhang Wang
getBytesIn and getBytesOut in BrokerAllTopicStat On Mon, Mar 31, 2014 at 11:12 AM, Arnaud Lawson wrote: > Ok thanks Guozhang . So What would the metrics for the in-traffic and > out-traffic message count be on this page (this page shows all existing jmx > metrics)? > > https://cwiki.apache.org/c

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Chi Hoang
We've used bytes written as in, and bytes read as out. Chi On Mon, Mar 31, 2014 at 11:12 AM, Arnaud Lawson wrote: > Ok thanks Guozhang . So What would the metrics for the in-traffic and > out-traffic message count be on this page (this page shows all existing jmx > metrics)? > > https://cwiki.a

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Arnaud Lawson
Ok thanks Guozhang . So What would the metrics for the in-traffic and out-traffic message count be on this page (this page shows all existing jmx metrics)? https://cwiki.apache.org/confluence/display/KAFKA/Operations#Operations-Monitoring On Mon, Mar 31, 2014 at 1:01 PM, Guozhang Wang wrote: >

Re: Kafka Queue Depth Metrics

2014-03-31 Thread Guozhang Wang
Hi Arnaud, Currently we do not have metric(s) for the number of messages on the kafka broker, though we do have the metrics for the in-traffic and out-traffic message count. One work-around would be the ConsumerOffsetChecker tool, for which it shows the last offset of messages on the brokers v.s.

Kafka Queue Depth Metrics

2014-03-31 Thread Arnaud Lawson
Hi, Does anyone know what the JMX metric(s) are for determining the depth of the kafka queue? I basically want to know the amount of messages that are on the queue at a certain point in time. Please let me know if there is a way to find that out. Thanks. -- Arnaud Lawson , Systems Operations E