Thanks, Otis. I actually already have a reporting and alerting infrastructure.
I mainly wanted to confirm that parsing the output of the offset checker is the recommended practice for reporting consumer group offsets. Is this the case? If so, I wanted to find out if any work is under way to make this more streamlined. On Mon, Apr 6, 2015 at 10:05 PM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Hi, > > We track our Consumer Lag and all other Kafka metrics with SPM > <http://sematext.com/spm> whose agent collects this info from Kafka > Consumers. Here's a chart of that: > https://apps.sematext.com/spm-reports/s/mcBHvsXwgj -- this public chart > doesn't show all filtering option, but you can filter lag by host, client > id, topic, and partition. > > Otis > -- > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > Solr & Elasticsearch Support * http://sematext.com/ > > > On Mon, Apr 6, 2015 at 6:14 PM, Kyle Banker <kyleban...@gmail.com> wrote: > > > What is the best practice for reporting the lag on individual consumer > > groups (e.g., to Graphite)? > > > > A recent form post (http://search-hadoop.com/m/4TaT4x9qWm1) seems to > > indicate that parsing the output of the consumer offset checker tool and > > reporting that independently is what folks do. Is there a better way? > > > > Are there any plans for the Kafka brokers themselves to do this reporting > > (e.g., the master for each partition reports lag for all of its consumer > > groups)? > > > > Many thanks in advance. > > >
