Thanks a lot Joel.
On Tue, Oct 22, 2013 at 11:27 PM, Joel Koshy <jjkosh...@gmail.com> wrote: > > > (1)"kafka.network":type="RequestChannel",name="Processor-0-ResponseQueueSize", > > > "kafka.network":type="RequestChannel",name="Processor-1-ResponseQueueSize", > > > "kafka.network":type="RequestChannel",name="Processor-2-ResponseQueueSize" > > Im not getting what these Processor in (1) are actually for? > > Kafka's request handling utilizes a number of processor "threads" and > responses are added to the above queue before sending to the client. > This diagram may help: > https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Internals > > > > > (2)"kafka.server":type="FetcherLagMetrics",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-demo-0-ConsumerLag" > > This reports how far behind that replica is. The format is > > ReplicaFetcherThread-<fetcherid>-<leader-broker-id>-<leaderbrokerinfo>-<topic>-<partition>-ConsumerLag > > (Fetcher id is typically zero (since the default number of replica > fetchers is one).) > > > > > > (3)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-1-host_moni-192-168-145-119-port_9092-RequestsPerSec" > > > (4)"kafka.server":type="FetcherStats",name="ReplicaFetcherThread-0-2-host_moni-192-168-145-176-port_9092-BytesPerSec" > > The Mbeans (2),(3) and (4) are getting generated on my 3rd broker - > > host_moni-192.168.145.67,kafka running on same port-9092.Any reason for > > these. > > Requests per sec/byte rate for replica fetchers, but for all fetches > to the indicated leader brokers (1 and 2). > > > > I am getting some more Mbeans which are not mentioned here- > > http://kafka.apache.org/documentation.html#monitoring > > > Kafka exposes a *lot* of mbeans - not all of them are listed on that > page that you are referring to. I think the goal of the table was to > provide a minimal set of critical metrics that people may want to set > up monitoring/alerting for. > > Thanks, > > Joel > > > > > On Tue, Oct 22, 2013 at 11:01 PM, Monika Garg <gargmon...@gmail.com> > wrote: > > > >> Thanks Jun. > >> > >> > >> > >> > >> On Tue, Oct 22, 2013 at 8:51 PM, Jun Rao <jun...@gmail.com> wrote: > >> > >>> When a broker goes down, we have an optimization in the controller that > >>> proactively shrinks ISR. The ISR shrink rate only measures the rate > that > >>> the leader shrinks ISR and doesn't reflect the shrinking by the > >>> controller. > >>> In general, having ISR shrink rate stay at 0 is a good thing since it > >>> means > >>> that the followers can always keep up. > >>> > >>> Thanks, > >>> > >>> Jun > >>> > >>> > >>> On Tue, Oct 22, 2013 at 12:55 AM, Monika Garg <gargmon...@gmail.com> > >>> wrote: > >>> > >>> > Thanks a lot Jun. > >>> > > >>> > Now im getting the values in ISR expansion rate,whenever broker goes > >>> down > >>> > and then up. > >>> > > >>> > But still ISR shrink rate is always having value 0. > >>> > > >>> > what I have done is: > >>> > > >>> > I made 3 node cluster.Than created 5 partitions with replica=3. > >>> > partitions got distributed on 1,2,3 brokers with isr= 1,2,3, and > >>> replica = > >>> > 1,2,3 > >>> > broker -3 is leader for 2 partitions. > >>> > > >>> > then i stopped one broker(say 3),then isr = 1,2 and replica = 1,2,3. > >>> > In this case I waited for some time to get the ISR shrink value to > get > >>> > updated in my jconsole,but value remained 0 for all attributes of ISR > >>> > shrink rate. > >>> > > >>> > Then I started broker(say 3) that i stopped earlier,what I saw is: > >>> > > >>> > ISR expansion rate has the values as: > >>> > > >>> > count -2 (on both other brokers 1,2) > >>> > oneminuterate - 0.012 etc. > >>> > > >>> > then I again stopped broker-1,still isr shrink rate not getting any > >>> > value.Can you please check? > >>> > > >>> > One more thing if you can please have a look: > >>> > How the other attributes like FifteenMinuteRate,OneMinuteRate etc are > >>> > continuosly changing for any Mbean. > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > > >>> > On Tue, Oct 22, 2013 at 9:00 AM, Jun Rao <jun...@gmail.com> wrote: > >>> > > >>> > > Updated the description in > >>> > > http://kafka.apache.org/documentation.html#monitoring. Does that > help > >>> > make > >>> > > things clearer? > >>> > > > >>> > > Thanks, > >>> > > > >>> > > Jun > >>> > > > >>> > > > >>> > > On Mon, Oct 21, 2013 at 9:43 AM, Monika Garg <gargmon...@gmail.com > > > >>> > wrote: > >>> > > > >>> > > > Thanks for replying Neha.It helped me a lot in getting the things > >>> more > >>> > > > clear. > >>> > > > I just have some doubts for the value of below Mbeans,please > have a > >>> > look: > >>> > > > (1)ISR expansion > >>> > > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager" > >>> > > > ,non-zero only during broker startup. > >>> > > > Does the property (1) mean that,it will be become zero when any > >>> broker > >>> > > (in > >>> > > > a 2 node cluster)is started,and will be non-zero again if both > the > >>> > > brokers > >>> > > > are stopped. > >>> > > > (2)ISR shrink > >>> > > > > rate::"kafka.server":name="ISRShrinksPerSec",type="ReplicaManager" > >>> > > > , 0 > >>> > > > Will this property(2) be always be zero? > >>> > > > Thanks. > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > On Mon, Oct 21, 2013 at 8:35 PM, Neha Narkhede < > >>> > neha.narkh...@gmail.com > >>> > > > >wrote: > >>> > > > > >>> > > > > You may find this useful - > >>> > > > > http://kafka.apache.org/documentation.html#monitoring > >>> > > > > > >>> > > > > Let us know how we can improve the documentation further. > >>> > > > > > >>> > > > > Thanks, > >>> > > > > Neha > >>> > > > > > >>> > > > > > >>> > > > > On Mon, Oct 21, 2013 at 5:42 AM, Monika Garg < > >>> gargmon...@gmail.com> > >>> > > > wrote: > >>> > > > > > >>> > > > > > Hi, > >>> > > > > > > >>> > > > > > I am getting so many Mbeans from My JMX like > >>> > > kafka.server,kafka.network > >>> > > > > > etc. > >>> > > > > > > >>> > > > > > There are of different types and have a lot of attributes. > >>> > > > > > > >>> > > > > > I am trying to get when the attribute value for any Mbean is > >>> > getting > >>> > > > > > changed. > >>> > > > > > > >>> > > > > > But no helpful docs I am getting for it on google. > >>> > > > > > > >>> > > > > > Even the terms used like > >>> > > > > > > >>> > kafka.server":type="FetchRequestPurgatory",name="PurgatorySize",here > >>> > > > > > how/when the value of PurgatorySize get change.No clue for > >>> this. > >>> > > > > > Similar to this there are a lot more values like > >>> > > > > > > >>> > > > > > when a topic is created,there is 554 values in kafka.network > >>> Mbean. > >>> > > > > > > >>> > > > > > Please any one can share any doc that explains the Mbeans for > >>> > > > Kafka-0.8. > >>> > > > > > > >>> > > > > > It will be really helpful for me. > >>> > > > > > Thanks. > >>> > > > > > > >>> > > > > > -- > >>> > > > > > *Moni* > >>> > > > > > > >>> > > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > -- > >>> > > > *Moniii* > >>> > > > > >>> > > > >>> > > >>> > > >>> > > >>> > -- > >>> > *Moniii* > >>> > > >>> > >> > >> > >> > >> -- > >> *Moniii* > >> > > > > > > > > -- > > *Moniii* > -- *Moniii*
