Hi All, I updated the proposal to include the GlobalParitionCount metric. The proposal for KIP-168 can be found here ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-168%3A+Add+GlobalTopicCount+and+GlobalPartitionCountmetric+per+cluster )
Best, Abhishek On Mon, Jul 17, 2017 at 2:06 PM, Abhishek Mendhekar < abhishek.mendhe...@gmail.com> wrote: > Hi Becket, > > Thank you for the comments. > > Vahid also requested a TotalPartitionCount metric and I guess the argument > of "aggregating this metric across all brokers in a large cluster can be > cumbersome and inefficient" is pretty convincing. I will add this to the > proposal and call this metric "GlobalParitionCount" (keeping it consistent > with GlobalTopicCount). > > Best, > Abhishek > > On Mon, Jul 17, 2017 at 1:37 PM, Becket Qin <becket....@gmail.com> wrote: > >> Hi Abhishek, >> >> Sorry for the late comments. I feel that the TotalPartitionCount seems a >> convenient metric to have. It is true that one can add the partition count >> on each broker and get the total partition count, but it is more work and >> in a large cluster with hundreds of brokers it seems not that efficient >> either. Having both the total topic count and total partition count also >> seem more symmetric. >> >> Thanks, >> >> Jiangjie (Becket) Qin >> >> On Wed, Jul 12, 2017 at 10:00 AM, Abhishek Mendhekar < >> abhishek.mendhe...@gmail.com> wrote: >> >> > Hi Vahid, >> > >> > Thanks for the prompt response. >> > >> > Since there may be redundant metrics let's defer the total number of >> > partition metric from the controller for now. >> > >> > Thanks, >> > Abhishek >> > >> > >> > On Mon, Jul 10, 2017 at 2:18 PM, Vahid S Hashemian < >> > vahidhashem...@us.ibm.com> wrote: >> > >> > > Hi Abhishek, >> > > >> > > Thanks for your response. >> > > Even though, as you suggested, this metric can be calculated using >> other >> > > present metrics, I still feel that adding the 'number of partitions' >> > > metrics helps in better representing a Kafka cluster. >> > > I don't feel too strong about it though (in case it is considered a >> > > redundant metrics based on your argument). >> > > >> > > Thanks. >> > > --Vahid >> > > >> > > >> > > >> > > >> > > From: Abhishek Mendhekar <abhishek.mendhe...@gmail.com> >> > > To: dev@kafka.apache.org >> > > Date: 07/10/2017 01:43 PM >> > > Subject: Re: [DISCUSS] KIP-168: Add TotalTopicCount metric per >> > > cluster >> > > >> > > >> > > >> > > Hi Vahid, >> > > >> > > Thanks for the response. >> > > >> > > Kafka currently has a partition leader count metric per broker. The >> total >> > > partition count metric can be built by summing the partition leader >> > > count across the cluster. This information may not be consistent but >> > it'll >> > > be eventually consistent in steady state. >> > > >> > > But if you think it'll be useful to add this metric in relation with >> the >> > > offline partition count to track them in tandem, then I guess it >> might be >> > > useful. Also, this will be consistent as it is coming from the >> > controller. >> > > Let me know your thoughts on this. >> > > >> > > Best, >> > > Abhishek >> > > >> > > >> > > On Wed, Jun 28, 2017 at 9:05 AM, Vahid S Hashemian < >> > > vahidhashem...@us.ibm.com> wrote: >> > > >> > > > Hi Abhishek, >> > > > >> > > > Thanks for the KIP and sorry for posting late on this thread. >> > > > >> > > > The proposal looks good to me. >> > > > I wonder if it would be useful to also add a partition count metric. >> > > > It seems there is already an offline partition count metric in the >> > > > controller, but no total partition count. >> > > > >> > > > Thanks. >> > > > --Vahid >> > > > >> > > > >> > > > >> > > > From: Abhishek Mendhekar <abhishek.mendhe...@gmail.com> >> > > > To: dev@kafka.apache.org >> > > > Date: 06/16/2017 09:09 AM >> > > > Subject: [DISCUSS] KIP-168: Add TotalTopicCount metric per >> > > cluster >> > > > >> > > > >> > > > >> > > > Hi Kafka Dev, >> > > > >> > > > I created KIP-168 to propose adding a metric to emit total topic >> count >> > > > in a cluster. The metric will be emited by the controller. >> > > > >> > > > The KIP can be found here >> > > > ( >> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- >> > > > 168%3A+Add+TotalTopicCount+metric+per+cluster >> > > > ) >> > > > and the assciated JIRA improvement is KAFKA-5461 >> > > > (https://issues.apache.org/jira/browse/KAFKA-5461) >> > > > >> > > > Appreciate all the comments. >> > > > >> > > > Best, >> > > > >> > > > Abhishek >> > > > >> > > > >> > > > >> > > > >> > > > >> > > >> > > >> > > -- >> > > Abhishek Mendhekar >> > > abhishek.mendhe...@gmail.com | 818.263.7030 >> > > >> > > >> > > >> > > >> > > >> > >> > >> > -- >> > Abhishek Mendhekar >> > abhishek.mendhe...@gmail.com | 818.263.7030 >> > >> > > > > -- > Abhishek Mendhekar > abhishek.mendhe...@gmail.com | 818.263.7030 <(818)%20263-7030> > -- Abhishek Mendhekar abhishek.mendhe...@gmail.com | 818.263.7030
