Found out about the following: JMX -> “kafka.server” -> (various FeatcherThread)-ConsumerLag
this sounds like the latency, in time. is it so? thanks, rob On May 17, 2013, at 11:49 AM, "Withers, Robert" <robert.with...@dish.com> wrote: > Could you add some JMX stats for us, then? > > - Queue length, by group offset vs lastOffset > - latency between produce and consume, again by group > > Thanks, > > > Rob Withers > Staff Analyst/Developer > o: (720) 514-8963 > c: (571) 262-1873 > > > > -----Original Message----- > From: Jun Rao [mailto:jun...@gmail.com] > Sent: Friday, May 17, 2013 8:52 AM > To: users@kafka.apache.org > Subject: Re: API to to query messages amount under one topic > > For monitoring, we have jmxs on the broker for both the message and the byte > rate. > > Thanks, > > Jun > > > On Thu, May 16, 2013 at 10:04 PM, Rob Withers <reefed...@gmail.com> wrote: > >> Immediately monitoring. Later, possible thresholding to evoke a >> reconfiguration of the number of partitions into a new topic and >> migrate message index/logs and redirect pubs/subs to the new topic, >> during a traffic spike. >> >>> -----Original Message----- >>> From: Jun Rao [mailto:jun...@gmail.com] >>> Sent: Thursday, May 16, 2013 10:00 PM >>> To: users@kafka.apache.org >>> Subject: Re: API to to query messages amount under one topic >>> >>> What do you plan to use this information for? >>> >>> Thanks, >>> >>> Jun >>> >>> >>> On Thu, May 16, 2013 at 5:57 AM, Withers, Robert >>> <robert.with...@dish.com>wrote: >>> >>>> Any possibility for a queuedMessageCount(topic, partitionNumber) >>> protocol? >>>> >>>> thanks, >>>> rob >>>> ________________________________________ >>>> From: Jun Rao [jun...@gmail.com] >>>> Sent: Wednesday, May 15, 2013 10:59 PM >>>> To: users@kafka.apache.org >>>> Subject: Re: API to to query messages amount under one topic >>>> >>>> In 0.8, you can get the earliest and the latest offset using the >>>> getOffsetBefore api. The difference btw the two gives the number >>>> of messages on the broker. >>>> >>>> Thanks, >>>> >>>> Jun >>>> >>>> >>>> On Wed, May 15, 2013 at 6:05 PM, Sining Ma <sinin...@aol.com> wrote: >>>> >>>>> Hi, >>>>> Is there any APIs in kafka that I can use to query how many >>>>> messages >>>> there >>>>> are under one topic? >>>>> This topic already exist in Kafka server, and producer is >>>>> sending >>>> messages >>>>> to one topic in kafka server >>>>> >>>>> >>>>> -- Regards >>>>> Sining Ma >>>>> >>>>> >>>>> >>>> >> >>
