Re: consumer lag metric

2015-02-17 Thread tao xiao
Thanks Todd. that will work On Tue, Feb 17, 2015 at 10:31 PM, Todd Palino wrote: > In order to do that, you'll need to run it and parse the output, and then > emit it to your metrics system of choice. This is essentially what I do - I > have a monitoring application which runs every minute and p

Re: consumer lag metric

2015-02-17 Thread Todd Palino
In order to do that, you'll need to run it and parse the output, and then emit it to your metrics system of choice. This is essentially what I do - I have a monitoring application which runs every minute and pulls the offsets for a select set of topics and consumers, and then packages up the metric

Re: consumer lag metric

2015-02-16 Thread tao xiao
Thank you Todd for your detailed explanation. Currently I export all metrics to graphite using the reporter configuration. is there a way I can do similar thing with offset checker? On Mon, Feb 16, 2015 at 4:21 PM, Todd Palino wrote: > The reason for this is the mechanic by which each of the lag

Re: consumer lag metric

2015-02-16 Thread Todd Palino
The reason for this is the mechanic by which each of the lags are calculated. MaxLag (and the FetcherLagMetric) are calculated by the consumer itself using the difference between the offset it knows it is at, and the offset that the broker has as the end of the partition. The offset checker, howeve

Re: consumer lag metric

2015-02-14 Thread tao xiao
Thanks Joel. But I discover that both MaxLag and FetcherLagMetrics are always much smaller than the lag shown in offset checker. any reason? On Sat, Feb 14, 2015 at 7:22 AM, Joel Koshy wrote: > There are FetcherLagMetrics that you can take a look at. However, it > is probably easiest to just mon

Re: consumer lag metric

2015-02-13 Thread Joel Koshy
There are FetcherLagMetrics that you can take a look at. However, it is probably easiest to just monitor MaxLag as that reports the maximum of all the lag metrics. On Fri, Feb 13, 2015 at 05:03:28PM +0800, tao xiao wrote: > Hi team, > > Is there a metric that shows the consumer lag of a particula

consumer lag metric

2015-02-13 Thread tao xiao
Hi team, Is there a metric that shows the consumer lag of a particular consumer group? similar to what offset checker provides -- Regards, Tao