The way I do it now is:
For each combination of topic name and partition number, I get
LogEndOffset and LogStartOffset from the JMX metrics, and do the
difference between them. I believe this will reflect the queue lengths.
Does that sound reasonable?
Second issue - I've found something call
Florin,
What may be of use is seeing the "consumer lag”. This is the offset position of
the latest message consumed by the consumer group, per partition, vs. the last
offset of the topic. So if a lag is 3, that means there are 3 unconsumed
messages by the consumer group.
I’m unsure where it wo
We have several producers and several consumers connected via Kafka -
basically, it's batch processing jobs that are created on demand, placed
on Kafka in several queues, and the batch processors pick them up via
Kafka and process them one by one.
I want to visualize and monitor the lengths of
