Zakee, It would be useful to get the following.
kafka.network:name=RequestQueueSize,type=RequestChannel kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics Thanks, Jun On Thu, Feb 26, 2015 at 2:17 PM, Zakee <kzak...@netzero.net> wrote: > Hi Jun, > > There are too many KPIs generated by kafka that I can’t monitor them all, > I have monitoring a filtered list that I can understand so far. > > Which keys are you talking of? Currently I am not monitoring any of below > keys but will add only those that are useful. > > kafka.network:name=RequestQueueSize,type=RequestChannel > > kafka.network:name=RequestQueueTimeMs,request=ConsumerMetadata,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=ControlledShutdown,type=RequestMetrics > kafka.network:name=RequestQueueTimeMs,request=Fetch,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=FetchConsumer,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=FetchFollower,type=RequestMetrics > kafka.network:name=RequestQueueTimeMs,request=Heartbeat,type=RequestMetrics > kafka.network:name=RequestQueueTimeMs,request=JoinGroup,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=LeaderAndIsr,type=RequestMetrics > kafka.network:name=RequestQueueTimeMs,request=Metadata,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=OffsetCommit,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=OffsetFetch,type=RequestMetrics > kafka.network:name=RequestQueueTimeMs,request=Offsets,type=RequestMetrics > kafka.network:name=RequestQueueTimeMs,request=Produce,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=StopReplica,type=RequestMetrics > > kafka.network:name=RequestQueueTimeMs,request=UpdateMetadata,type=RequestMetrics > > > Thanks, > Zakee > > > On Feb 26, 2015, at 9:10 AM, Jun Rao <j...@confluent.io> wrote: > > > > That may be enough. What's the RequestQueueSize and RequestQueueTimeMs? > > > > Thanks, > > > > Jun > > > > On Wed, Feb 25, 2015 at 10:24 PM, Zakee <kzak...@netzero.net> wrote: > > > >> Well currently I have configured 14 thread for both io and network. Do > you > >> think we should consider more? > >> > >> Thanks > >> -Zakee > >> > >> On Wed, Feb 25, 2015 at 6:22 PM, Jun Rao <j...@confluent.io> wrote: > >> > >>> Then you may want to consider increasing num.io.threads > >>> and num.network.threads. > >>> > >>> Thanks, > >>> > >>> Jun > >>> > >>> On Tue, Feb 24, 2015 at 7:48 PM, Zakee <kzak...@netzero.net> wrote: > >>> > >>>> Similar pattern for that too. Mostly hovering below. > >>>> > >>>> -Zakee > >>>> > >>>> On Tue, Feb 24, 2015 at 2:43 PM, Jun Rao <j...@confluent.io> wrote: > >>>> > >>>>> What about RequestHandlerAvgIdlePercent? > >>>>> > >>>>> Thanks, > >>>>> > >>>>> Jun > >>>>> > >>>>> On Mon, Feb 23, 2015 at 8:47 PM, Zakee <kzak...@netzero.net> wrote: > >>>>> > >>>>>> Hi Jun, > >>>>>> > >>>>>> With ~100G of data being pushed per hour across 35 topics > >>>>>> (replication-factor 3), the NetworkProcessorAvgIdlePercent is > >> mostly > >>>>>> showing below 0.5 sometimes when the producers send on a high rate. > >>>>>> > >>>>>> Thanks > >>>>>> -Zakee > >>>>>> > >>>>>> On Sun, Feb 22, 2015 at 10:29 PM, Jun Rao <j...@confluent.io> > >> wrote: > >>>>>> > >>>>>>> What kind of load do you have on the brokers? On an idle cluster > >>>> (just > >>>>>>> fetch requests from the follower replicas), I > >>>>>>> saw NetworkProcessorAvgIdlePercent at about 97%. > >>>>>>> > >>>>>>> Thanks, > >>>>>>> > >>>>>>> Jun > >>>>>>> > >>>>>>> On Thu, Feb 19, 2015 at 5:19 PM, Zakee <kzak...@netzero.net> > >>> wrote: > >>>>>>> > >>>>>>>> Jun, > >>>>>>>> > >>>>>>>> I am already using the latest release 0.8.2.1. > >>>>>>>> > >>>>>>>> -Zakee > >>>>>>>> > >>>>>>>> On Thu, Feb 19, 2015 at 2:46 PM, Jun Rao <j...@confluent.io> > >>> wrote: > >>>>>>>> > >>>>>>>>> Could you try the 0.8.2.1 release being voted on now? It > >> fixes > >>> a > >>>>> CPU > >>>>>>>> issue > >>>>>>>>> and should reduce the CPU load in network thread. > >>>>>>>>> > >>>>>>>>> Thanks, > >>>>>>>>> > >>>>>>>>> Jun > >>>>>>>>> > >>>>>>>>> On Thu, Feb 19, 2015 at 11:54 AM, Zakee <kzak...@netzero.net > >>> > >>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Kafka documentation recommends <0.3 for above metric. I > >>> assume > >>>>>>>> processor > >>>>>>>>> is > >>>>>>>>>> busier if this goes below 0.3 and obviously it being < 0.3 > >>> for > >>>>> long > >>>>>>>> does > >>>>>>>>>> not seem to be a good sign. > >>>>>>>>>> > >>>>>>>>>> What should be our criteria to raise an alert, I though it > >>>> should > >>>>>> be > >>>>>>>>> when > >>>>>>>>>> its value goes below 0.3. However, the value seems to be > >>> below > >>>>> 0.3 > >>>>>> a > >>>>>>>> lot > >>>>>>>>> of > >>>>>>>>>> the times, almost always if we take samples every five > >> mins. > >>>> What > >>>>>>>> should > >>>>>>>>> be > >>>>>>>>>> the threshold to raise an alarm ? > >>>>>>>>>> > >>>>>>>>>> What would be the impact of having this below 0.3 or even > >>> zero > >>>>> like > >>>>>>>> most > >>>>>>>>> of > >>>>>>>>>> the times? > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> -Zakee > >>>>>>>>>> > >> ____________________________________________________________ > >>>>>>>>>> How Old Men Tighten Skin > >>>>>>>>>> 63 Year Old Man Shares DIY Skin Tightening Method You Can > >> Do > >>>> From > >>>>>>> Home > >>>>>>>>>> > >>>>>>>> > >>>>>> > >>>> > >> > http://thirdpartyoffers.netzero.net/TGL3231/54e63f5bda4c23f5b6560st02vuc > >>>>>>>>> ____________________________________________________________ > >>>>>>>>> 8% Annuity Return Secret > >>>>>>>>> Earn Guaranteed Income for Life! Compare Rates Today. > >>>>>>>>> > >>>>>>> > >>>>> > >>> > http://thirdpartyoffers.netzero.net/TGL3255/54e6782bcbe78782b37bdmp15duc > >>>>>>>> > >>>>>>> ____________________________________________________________ > >>>>>>> High School Yearbooks > >>>>>>> View Class Yearbooks Online Free. Reminisce & Buy a Reprint > >> Today! > >>>>>>> > >>>>> > >>> > http://thirdpartyoffers.netzero.net/TGL3255/54eac962abad049627e56mp03duc > >>>>>> > >>>>> ____________________________________________________________ > >>>>> Extended Stay America > >>>>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed. > >>>>> > >>> > http://thirdpartyoffers.netzero.net/TGL3255/54ed04fb9538c4fb6188mp15duc > >>>> > >>> ____________________________________________________________ > >>> Extended Stay America > >>> Official Site. Free WIFI, Kitchens. Our best rates here, guaranteed. > >>> > http://thirdpartyoffers.netzero.net/TGL3255/54ee8dd7e1cd7dd777e8mp03duc > >> > > ____________________________________________________________ > > Protect what matters > > Floods can happen anywhere. Learn your risk and find an agent today. > > http://thirdpartyoffers.netzero.net/TGL3255/54ef551637a8b5514471fmp01duc > >
