Hi Becket, Thank you for the comments.
Vahid also requested a TotalPartitionCount metric and I guess the argument of "aggregating this metric across all brokers in a large cluster can be cumbersome and inefficient" is pretty convincing. I will add this to the proposal and call this metric "GlobalParitionCount" (keeping it consistent with GlobalTopicCount). Best, Abhishek On Mon, Jul 17, 2017 at 1:37 PM, Becket Qin <becket....@gmail.com> wrote: > Hi Abhishek, > > Sorry for the late comments. I feel that the TotalPartitionCount seems a > convenient metric to have. It is true that one can add the partition count > on each broker and get the total partition count, but it is more work and > in a large cluster with hundreds of brokers it seems not that efficient > either. Having both the total topic count and total partition count also > seem more symmetric. > > Thanks, > > Jiangjie (Becket) Qin > > On Wed, Jul 12, 2017 at 10:00 AM, Abhishek Mendhekar < > abhishek.mendhe...@gmail.com> wrote: > > > Hi Vahid, > > > > Thanks for the prompt response. > > > > Since there may be redundant metrics let's defer the total number of > > partition metric from the controller for now. > > > > Thanks, > > Abhishek > > > > > > On Mon, Jul 10, 2017 at 2:18 PM, Vahid S Hashemian < > > vahidhashem...@us.ibm.com> wrote: > > > > > Hi Abhishek, > > > > > > Thanks for your response. > > > Even though, as you suggested, this metric can be calculated using > other > > > present metrics, I still feel that adding the 'number of partitions' > > > metrics helps in better representing a Kafka cluster. > > > I don't feel too strong about it though (in case it is considered a > > > redundant metrics based on your argument). > > > > > > Thanks. > > > --Vahid > > > > > > > > > > > > > > > From: Abhishek Mendhekar <abhishek.mendhe...@gmail.com> > > > To: dev@kafka.apache.org > > > Date: 07/10/2017 01:43 PM > > > Subject: Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per > > > cluster > > > > > > > > > > > > Hi Vahid, > > > > > > Thanks for the response. > > > > > > Kafka currently has a partition leader count metric per broker. The > total > > > partition count metric can be built by summing the partition leader > > > count across the cluster. This information may not be consistent but > > it'll > > > be eventually consistent in steady state. > > > > > > But if you think it'll be useful to add this metric in relation with > the > > > offline partition count to track them in tandem, then I guess it might > be > > > useful. Also, this will be consistent as it is coming from the > > controller. > > > Let me know your thoughts on this. > > > > > > Best, > > > Abhishek > > > > > > > > > On Wed, Jun 28, 2017 at 9:05 AM, Vahid S Hashemian < > > > vahidhashem...@us.ibm.com> wrote: > > > > > > > Hi Abhishek, > > > > > > > > Thanks for the KIP and sorry for posting late on this thread. > > > > > > > > The proposal looks good to me. > > > > I wonder if it would be useful to also add a partition count metric. > > > > It seems there is already an offline partition count metric in the > > > > controller, but no total partition count. > > > > > > > > Thanks. > > > > --Vahid > > > > > > > > > > > > > > > > From: Abhishek Mendhekar <abhishek.mendhe...@gmail.com> > > > > To: dev@kafka.apache.org > > > > Date: 06/16/2017 09:09 AM > > > > Subject: [DISCUSS] KIP-168: Add TotalTopicCount metric per > > > cluster > > > > > > > > > > > > > > > > Hi Kafka Dev, > > > > > > > > I created KIP-168 to propose adding a metric to emit total topic > count > > > > in a cluster. The metric will be emited by the controller. > > > > > > > > The KIP can be found here > > > > ( > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > > > > 168%3A+Add+TotalTopicCount+metric+per+cluster > > > > ) > > > > and the assciated JIRA improvement is KAFKA-5461 > > > > (https://issues.apache.org/jira/browse/KAFKA-5461) > > > > > > > > Appreciate all the comments. > > > > > > > > Best, > > > > > > > > Abhishek > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Abhishek Mendhekar > > > abhishek.mendhe...@gmail.com | 818.263.7030 > > > > > > > > > > > > > > > > > > > > > -- > > Abhishek Mendhekar > > abhishek.mendhe...@gmail.com | 818.263.7030 > > > -- Abhishek Mendhekar abhishek.mendhe...@gmail.com | 818.263.7030
