We monitor producers or for that matter any process/service using JMX
metrics. Every server and service in LinkedIn sends metrics in a Kafka
message to a metrics Kafka cluster. We have subscribers that connect to the
metrics cluster to index that data in RRDs.
Our aim is to expose all important me
Sorry I want to know you want to monitor kafka producers or kafka brokers
and zookeepers ?
It's seems you will want to monitor monitor Exceptions eg Leader Not Found,
Queue is full, resend fail etc are kafka cluster
2014-06-25 8:20 GMT+08:00 Bhavesh Mistry :
> We use Kafka as Transport Layer
We use Kafka as Transport Layer to transport application logs. How do we
monitor Producers at large scales about 6000 boxes x 4 topic per box so
roughly 24000 producers (spread across multiple data center.. we have
brokers per DC). We do the monitoring based on logs. I have tried
intercepting lo
