Hi Jun, There are too many KPIs generated by kafka that I can’t monitor them all, I have monitoring a filtered list that I can understand so far.
Which keys are you talking of? Currently I am not monitoring any of below keys but will add only those that are useful. kafka.network:name=RequestQueueSize,type=RequestChannel kafka.network:name=RequestQueueTimeMs,request=ConsumerMetadata,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=ControlledShutdown,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=FetchConsumer,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=FetchFollower,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=Heartbeat,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=JoinGroup,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=LeaderAndIsr,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=Metadata,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=OffsetCommit,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=OffsetFetch,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=Offsets,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=StopReplica,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=UpdateMetadata,type=RequestMetrics Thanks, Zakee > On Feb 26, 2015, at 9:10 AM, Jun Rao <j...@confluent.io> wrote: > > That may be enough. What's the RequestQueueSize and RequestQueueTimeMs? > > Thanks, > > Jun > > On Wed, Feb 25, 2015 at 10:24 PM, Zakee <kzak...@netzero.net> wrote: > >> Well currently I have configured 14 thread for both io and network. Do you >> think we should consider more? >> >> Thanks >> -Zakee >> >> On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao <j...@confluent.io> wrote: >> >>> Then you may want to consider increasing num.io.threads >>> and num.network.threads. >>> >>> Thanks, >>> >>> Jun >>> >>> On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kzak...@netzero.net> wrote: >>> >>>> Similar pattern for that too. Mostly hovering below. >>>> >>>> -Zakee >>>> >>>> On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <j...@confluent.io> wrote: >>>> >>>>> What about RequestHandlerAvgIdlePercent? >>>>> >>>>> Thanks, >>>>> >>>>> Jun >>>>> >>>>> On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kzak...@netzero.net> wrote: >>>>> >>>>>> Hi Jun, >>>>>> >>>>>> With ~100G of data being pushed per hour across 35 topics >>>>>> (replication-factor 3), the NetworkProcessorAvgIdlePercent is >> mostly >>>>>> showing below 0.5 sometimes when the producers send on a high rate. >>>>>> >>>>>> Thanks >>>>>> -Zakee >>>>>> >>>>>> On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <j...@confluent.io> >> wrote: >>>>>> >>>>>>> What kind of load do you have on the brokers? On an idle cluster >>>> (just >>>>>>> fetch requests from the follower replicas), I >>>>>>> saw NetworkProcessorAvgIdlePercent at about 97%. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Jun >>>>>>> >>>>>>> On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kzak...@netzero.net> >>> wrote: >>>>>>> >>>>>>>> Jun, >>>>>>>> >>>>>>>> I am already using the latest release 0.8.2.1. >>>>>>>> >>>>>>>> -Zakee >>>>>>>> >>>>>>>> On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <j...@confluent.io> >>> wrote: >>>>>>>> >>>>>>>>> Could you try the 0.8.2.1 release being voted on now? It >> fixes >>> a >>>>> CPU >>>>>>>> issue >>>>>>>>> and should reduce the CPU load in network thread. >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> >>>>>>>>> Jun >>>>>>>>> >>>>>>>>> On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kzak...@netzero.net >>> >>>>> wrote: >>>>>>>>> >>>>>>>>>> Kafka documentation recommends <0.3 for above metric. I >>> assume >>>>>>>> processor >>>>>>>>> is >>>>>>>>>> busier if this goes below 0.3 and obviously it being < 0.3 >>> for >>>>> long >>>>>>>> does >>>>>>>>>> not seem to be a good sign. >>>>>>>>>> >>>>>>>>>> What should be our criteria to raise an alert, I though it >>>> should >>>>>> be >>>>>>>>> when >>>>>>>>>> its value goes below 0.3. However, the value seems to be >>> below >>>>> 0.3 >>>>>> a >>>>>>>> lot >>>>>>>>> of >>>>>>>>>> the times, almost always if we take samples every five >> mins. >>>> What >>>>>>>> should >>>>>>>>> be >>>>>>>>>> the threshold to raise an alarm ? >>>>>>>>>> >>>>>>>>>> What would be the impact of having this below 0.3 or even >>> zero >>>>> like >>>>>>>> most >>>>>>>>> of >>>>>>>>>> the times? >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -Zakee >>>>>>>>>> >> ____________________________________________________________ >>>>>>>>>> How Old Men Tighten Skin >>>>>>>>>> 63 Year Old Man Shares DIY Skin Tightening Method You Can >> Do >>>> From >>>>>>> Home >>>>>>>>>> >>>>>>>> >>>>>> >>>> >> http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc >>>>>>>>> ____________________________________________________________ >>>>>>>>> 8% Annuity Return Secret >>>>>>>>> Earn Guaranteed Income for Life! Compare Rates Today. >>>>>>>>> >>>>>>> >>>>> >>> http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc >>>>>>>> >>>>>>> ____________________________________________________________ >>>>>>> High School Yearbooks >>>>>>> View Class Yearbooks Online Free. Reminisce & Buy a Reprint >> Today! >>>>>>> >>>>> >>> http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc >>>>>> >>>>> ____________________________________________________________ >>>>> Extended Stay America >>>>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed. >>>>> >>> http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc >>>> >>> ____________________________________________________________ >>> Extended Stay America >>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed. >>> http://thirdpartyoffers.netzero.net/TGL3255/54ee8dd7e1cd7dd777e8mp03duc >> > ____________________________________________________________ > Protect what matters > Floods can happen anywhere. Learn your risk and find an agent today. > http://thirdpartyoffers.netzero.net/TGL3255/54ef551637a8b5514471fmp01duc
