I have a system which gathers logs from several clients.
The client logs are produced with a key specifying which client it is.
That way processing logs is guaranteed to be ordered by time per user.
Getting metrics per the topic is good and all, but getting metrics per key
allows you to analyze how many logs there are per client.
Of course one can claim the entity responsible for processing the logs can
expose those kinds of metrics.
But this only gives you the metrics on the already processed logs, instead
of merely reporting how much input per client got in.
I can create a different consumer group only for that purpose, but ideally
I would expect such information to be exposed by the broker itself.
Given the broker already calculates the message rate, doing the same per
key (while limiting the memory of course) wouldn't be unreasonable.

Reply via email to