Hi Chai-Ping, Thank you for bringing up this proposal. My team also has the need to have per partition metrics on traffic, which would be used by our partition rebalancer, I have a prototype running in test cluster that looks good so far.
Some thoughts on bringing this to larger scale: 1. Simply adding another dimension (tag) to BrokerTopicsMetrics might cause pressure on Jmx? For our cluster with 30,000+ partitions, 300+ topics, this might cause a explode in number of registered Mbeans. 2. We may consider 1) Create another BrokerPartitonsMetrics 2) Opt them out from JmxReporter Thanks, Xue On Wed, Mar 18, 2020 at 10:26 PM Chia-Ping Tsai <chia7...@apache.org> wrote: > hi Cheng > > seems the "Next KIP Number" is not synced :( > > I have updated my KIP number and corrected the "Next KIP Number" > > thanks for the reminder. > > On 2020/03/19 02:32:55, "Cheng Pan" <379377...@qq.com> wrote: > > Hi Chia-Ping, > > > > > > I'm afraid the KIP number is conflict. I created (KIP-581: Value of > optional null field which has default value) yesterday early, and insert it > into table [KIPs under discussion]. > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-581%3A+Value+of+optional+null+field+which+has+default+value > > > > > > Would you please choose another KIP number, maybe 583, I also see > KIP-582 discussion thread appears in the mail list, and insert into table > [KIPs under discussion]. > > > > > > Best Regards, > > Cheng Pan > > ------------------ Original ------------------ > > From: "Chia-Ping Tsai";<chia7...@apache.org>; > > Send time: Thursday, Mar 19, 2020 1:48 AM > > To: "dev"<dev@kafka.apache.org>; > > > > Subject: [DISCUSS] KIP-581: add tag "partition" to BrokerTopicMetrics > so as to observe the partition metrics on the same broker > > > > > > > > hi > > > > this ticket is about to records partition metrics rather than topic > metrics. It helps us to observe more precis metrics for specify partition. > The downside is that broker needs more space to keep metrics and the origin > metrics interface (tags) is broken since this ticket adds new tag > "partition=xxx" to it. The alternative is an new config flag used to enable > new metrics (of course, it is false by default). Also, we ought to provide > enough document to explain benefit and cost of new metrics. > > > > KIP-581 ( > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148642648 > ) > > > > JIRA (https://issues.apache.org/jira/browse/KAFKA-9730) > > > > Please take a look :) > > > > Best Regards, > > Chia-Ping >
