Based on PR feedback I've updated the KIP to align the configs between
clients and brokers.
Broker configs now start with "metrics.xxx" instead of "kafka.metrics.xxx",
in line with clients configs.
This is also more consistent with newer broker configs.
On Fri, Nov 8, 2019 at 12:23 PM Alexandre Du
Hello,
This can be very handy when dealing with large numbers of partitions on a
broker.
I was recently experimenting with a third-party monitoring framework which
provides a JMX collector [1] with the same mechanism to filter out the JMX
beans retrieved from Kafka.
When running a couple of tests
Sounds good, thanks.
Guozhang
On Fri, Nov 8, 2019 at 9:26 AM Xavier Léauté wrote:
> >
> > 1. I do feel there're similar needs for clients make JMX configurable.
> Some
> > context: in modules like Connect and Streams we have added / refactored a
> > large number of metrics so far [0, 1], and al
>
> 1. I do feel there're similar needs for clients make JMX configurable. Some
> context: in modules like Connect and Streams we have added / refactored a
> large number of metrics so far [0, 1], and although we've added a reporting
> level config [2] to clients, this is statically defined at code
Thanks Xavier for the KIP, I think it is a great idea to add to AK. A
couple of questions / comments:
1. I do feel there're similar needs for clients make JMX configurable. Some
context: in modules like Connect and Streams we have added / refactored a
large number of metrics so far [0, 1], and alt
>
> Since these configs will work with Kafka's own metrics library, will the
> configs be part of the clients' configurations? It would be good to point
> that out explicitly in the KIP.
>
Those configs are currently only at the broker level. If we feel this is
useful on the client as well, we cou
Hi Xavier,
That's certainly an option, however it does not solve the problem for our
users that still rely on JMX integration to collect metrics.
Absolutely.
We already provide the ability to write reporter plugins via the
MetricsReporter interface.
And rather than building a separate HTTP inter
>
> A follow-up question, maybe to list in the future work section as it's
> somewhat parallel to this KIP: have you thought about implementing a REST
> reporter for metrics?
That's certainly an option, however it does not solve the problem for our
users that still rely on JMX integration to coll
Hey Xavier,
Thanks for the explanation.
A follow-up question, maybe to list in the future work section as it's
somewhat parallel to this KIP: have you thought about implementing a REST
reporter for metrics? In my opinion it would be useful as you can write it
so that you directly query the registr
>
> How would the practical application look like if this was implemented?
>
One useful application is to hide partition-level metrics, some of which
may only be needed for debugging purposes.
> Would monitoring agents switch between the whitelist and blacklist
> periodically if they wanted to m
Hey Xavier,
Thank you for working on this. This KIP looks very good to me.
Since these configs will work with Kafka's own metrics library, will the
configs be part of the clients' configurations? It would be good to point
that out explicitly in the KIP.
I also second Viktor's question on what th
Hi Xavier,
How would the practical application look like if this was implemented?
Would monitoring agents switch between the whitelist and blacklist
periodically if they wanted to monitor every metrics?
I think we should make some usage recommendations.
Thanks,
Viktor
On Sun, Oct 27, 2019 at 3:3
Thanks Xavier.
I really like this proposal. Collecting JMX metrics in clusters with
100K partitions was nearly impossible due to the design of JMX and the
single lock mechanism. Yammer's limitations meant that any metric we
reported was exposed via JMX, so we couldn't have cheaper reporters
export
Hi All,
I wrote a short KIP to make the set of metrics exposed via JMX configurable.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-544%3A+Make+metrics+exposed+via+JMX+configurable
Let me know what you think.
Thanks,
Xavier
14 matches
Mail list logo
