Hi David,
these two metrics have two different objectives.
> kafka.server:type=BrokerTopicMetrics,name=TotalProduceRequestsPerSec <
****
Produce request rate per topic. Omitting 'topic=(...)' will yield the
all-topic rate.
*****
while
>kafka.network:type=RequestMetrics,name=RequestsPerSec,request={Produce|FetchConsumer|FetchFollower}
<
it is going to report as well requests per sec, however, you can filter and
select by Produce/Fetch from consumers or Fetch from Followers.
Note this is a metric collected at the network type, while the previous is
at the server level.
The first one will provide you with info per topic, which is a very
valuable asset in order to know more about what kind of usage clients are
doing from a topic perspective. Keep in mind the second one, will only give
you the global request per second.
So to your questions:
> Shouldn't both metrics indicate the same?
They indicate similar data at different levels, so they should not be far
away, however, some distance (but minimal) between the two could be
expected as they are collected at two different points.
> seems to be the accurate one matching the number of messages sent with no
batching.
In a good monitoring solution, you should have both metrics as one will
give you the overall load and the other the topic partition per broker view
(aka what are your users doing).
what I don't understand is the batching part of your question.
If you like to see messages In I would suggest you to use
> kafka.server:type=BrokerTopicMetrics,name=MessagesInPerSec <
Aggregate incoming message rate.
Sincerely
-- Pere
On Sat, Feb 18, 2023 at 12:17 AM David Ballano Fernandez <
dfernan...@demonware.net> wrote:
> Hi guys,
>
> I am having some confusion around 2 Kafka metrics:
>
> *Request rate.*
>
> kafka.network:type=RequestMetrics,name=RequestsPerSec,request={Produce|FetchConsumer|FetchFollower}
>
> *Produce request rate.*
> kafka.server:type=BrokerTopicMetrics,name=TotalProduceRequestsPerSec
>
>
> https://docs.confluent.io/platform/current/kafka/monitoring.html#server-metrics
>
> They seem to be tracking the same, but when I graph them and say I pick
> Producer. The numbers are different.
>
> and looks like
>
> *Produce request rate.*
> kafka.server:type=BrokerTopicMetrics,name=TotalProduceRequestsPerSec
>
> seems to be the accurate one matching the number of messages sent with no
> batching.
>
> Shouldn't both metrics indicate the same?
>
> Thanks!
>
--
Pere Urbon-Bayes
Software Architect
https://twitter.com/purbon
https://www.linkedin.com/in/purbon/
