Re: Kafka metrics to calculate number of messages in a topic

2021-08-18 Thread Eric Azama
That will get you a good approximation, but it's not guaranteed to be completely accurate. Offsets in Kafka are not guaranteed to be continuous. For topics with log compaction enabled, the removed records will leave (potentially very large) holes in the offsets. Even for topics without log compac

Kafka metrics to calculate number of messages in a topic

2021-08-09 Thread Dhirendra Singh
Hi All, I have a requirement to display the total number of messages in a topic in grafana dashboard. I am looking at the metrics exposed by kafka broker and came across the following metrics. kafka_log_log_logendoffset kafka_log_log_logstartoffset My understanding is that if I take the difference