> (1)"kafka.network":type="RequestChannel",name="Processor-0-ResponseQueueSize", > "kafka.network":type="RequestChannel",name="Processor-1-ResponseQueueSize", > "kafka.network":type="RequestChannel",name="Processor-2-ResponseQueueSize" > Im not getting what these Processor in (1) are actually for?
Kafka's request handling utilizes a number of processor "threads" and responses are added to the above queue before sending to the client. This diagram may help: https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals > (2)"kafka.server":type="FetcherLagMetrics",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-demo-0-ConsumerLag" This reports how far behind that replica is. The format is ReplicaFetcherThread-<fetcherid>-<leader-broker-id>-<leaderbrokerinfo>-<topic>-<partition>-ConsumerLag (Fetcher id is typically zero (since the default number of replica fetchers is one).) > > (3)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-RequestsPerSec" > (4)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-2-host_moni-192-168-145-176-port_9092-BytesPerSec" > The Mbeans (2),(3) and (4) are getting generated on my 3rd broker - > host_moni-192.168.145.67,kafka running on same port-9092.Any reason for > these. Requests per sec/byte rate for replica fetchers, but for all fetches to the indicated leader brokers (1 and 2). > I am getting some more Mbeans which are not mentioned here- > http://kafka.apache.org/documentation.html#monitoring > Kafka exposes a *lot* of mbeans - not all of them are listed on that page that you are referring to. I think the goal of the table was to provide a minimal set of critical metrics that people may want to set up monitoring/alerting for. Thanks, Joel > > On Tue, Oct 22, 2013 at 11:01 PM, Monika Garg <gargmon...@gmail.com> wrote: > >> Thanks Jun. >> >> >> >> >> On Tue, Oct 22, 2013 at 8:51 PM, Jun Rao <jun...@gmail.com> wrote: >> >>> When a broker goes down, we have an optimization in the controller that >>> proactively shrinks ISR. The ISR shrink rate only measures the rate that >>> the leader shrinks ISR and doesn't reflect the shrinking by the >>> controller. >>> In general, having ISR shrink rate stay at 0 is a good thing since it >>> means >>> that the followers can always keep up. >>> >>> Thanks, >>> >>> Jun >>> >>> >>> On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <gargmon...@gmail.com> >>> wrote: >>> >>> > Thanks a lot Jun. >>> > >>> > Now im getting the values in ISR expansion rate,whenever broker goes >>> down >>> > and then up. >>> > >>> > But still ISR shrink rate is always having value 0. >>> > >>> > what I have done is: >>> > >>> > I made 3 node cluster.Than created 5 partitions with replica=3. >>> > partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and >>> replica = >>> > 1,2,3 >>> > broker -3 is leader for 2 partitions. >>> > >>> > then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3. >>> > In this case I waited for some time to get the ISR shrink value to get >>> > updated in my jconsole,but value remained 0 for all attributes of ISR >>> > shrink rate. >>> > >>> > Then I started broker(say 3) that i stopped earlier,what I saw is: >>> > >>> > ISR expansion rate has the values as: >>> > >>> > count -2 (on both other brokers 1,2) >>> > oneminuterate - 0.012 etc. >>> > >>> > then I again stopped broker-1,still isr shrink rate not getting any >>> > value.Can you please check? >>> > >>> > One more thing if you can please have a look: >>> > How the other attributes like FifteenMinuteRate,OneMinuteRate etc are >>> > continuosly changing for any Mbean. >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <jun...@gmail.com> wrote: >>> > >>> > > Updated the description in >>> > > http://kafka.apache.org/documentation.html#monitoring. Does that help >>> > make >>> > > things clearer? >>> > > >>> > > Thanks, >>> > > >>> > > Jun >>> > > >>> > > >>> > > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <gargmon...@gmail.com> >>> > wrote: >>> > > >>> > > > Thanks for replying Neha.It helped me a lot in getting the things >>> more >>> > > > clear. >>> > > > I just have some doubts for the value of below Mbeans,please have a >>> > look: >>> > > > (1)ISR expansion >>> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager" >>> > > > ,non-zero only during broker startup. >>> > > > Does the property (1) mean that,it will be become zero when any >>> broker >>> > > (in >>> > > > a 2 node cluster)is started,and will be non-zero again if both the >>> > > brokers >>> > > > are stopped. >>> > > > (2)ISR shrink >>> > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager" >>> > > > , 0 >>> > > > Will this property(2) be always be zero? >>> > > > Thanks. >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > >>> > > > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede < >>> > neha.narkh...@gmail.com >>> > > > >wrote: >>> > > > >>> > > > > You may find this useful - >>> > > > > http://kafka.apache.org/documentation.html#monitoring >>> > > > > >>> > > > > Let us know how we can improve the documentation further. >>> > > > > >>> > > > > Thanks, >>> > > > > Neha >>> > > > > >>> > > > > >>> > > > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg < >>> gargmon...@gmail.com> >>> > > > wrote: >>> > > > > >>> > > > > > Hi, >>> > > > > > >>> > > > > > I am getting so many Mbeans from My JMX like >>> > > kafka.server,kafka.network >>> > > > > > etc. >>> > > > > > >>> > > > > > There are of different types and have a lot of attributes. >>> > > > > > >>> > > > > > I am trying to get when the attribute value for any Mbean is >>> > getting >>> > > > > > changed. >>> > > > > > >>> > > > > > But no helpful docs I am getting for it on google. >>> > > > > > >>> > > > > > Even the terms used like >>> > > > > > >>> > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here >>> > > > > > how/when the value of PurgatorySize get change.No clue for >>> this. >>> > > > > > Similar to this there are a lot more values like >>> > > > > > >>> > > > > > when a topic is created,there is 554 values in kafka.network >>> Mbean. >>> > > > > > >>> > > > > > Please any one can share any doc that explains the Mbeans for >>> > > > Kafka-0.8. >>> > > > > > >>> > > > > > It will be really helpful for me. >>> > > > > > Thanks. >>> > > > > > >>> > > > > > -- >>> > > > > > *Moni* >>> > > > > > >>> > > > > >>> > > > >>> > > > >>> > > > >>> > > > -- >>> > > > *Moniii* >>> > > > >>> > > >>> > >>> > >>> > >>> > -- >>> > *Moniii* >>> > >>> >> >> >> >> -- >> *Moniii* >> > > > > -- > *Moniii*
