Take a look into how Offset Requests are made and handled. You could do that to get the low and high watermarks for the stream. With the high and low watermark just subtract them to get total. This will be in the consumer api part of whatever language client your using. The specs for this are https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol#AGuideToTheKafkaProtocol-OffsetAPI
/******************************************* Joe Stein Founder, Principal Consultant Big Data Open Source Security LLC http://www.stealth.ly Twitter: @allthingshadoop ********************************************/ On Oct 5, 2014 8:35 AM, "Shlomi Hazan" <shl...@viber.com> wrote: actually this tool is not a 100% match to what I need, since it can only provide information on topics that have comsumers: Is there also another equivalent tool/method of querying topics that have no consumers ? in this case this tool will not help as it requires a group id as a mandatory parameter... On Sun, Oct 5, 2014 at 1:22 PM, Shlomi Hazan <shl...@viber.com> wrote: > Bingo. 10x!! > > On Wed, Oct 1, 2014 at 6:41 PM, chetan conikee <coni...@gmail.com> wrote: > >> The other method is via command line >> >> bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --group >> *groupName* >> --zkconnect *zkServer:2181* >> >> Refer : >> >> https://cwiki.apache.org/confluence/display/KAFKA/System+Tools#SystemTools-ConsumerOffsetChecker >> >> >> https://apache.googlesource.com/kafka/+/0.8.0-beta1-candidate1/core/src/main/scala/kafka/tools/ConsumerOffsetChecker.scala >> >> On Wed, Oct 1, 2014 at 8:28 AM, Gwen Shapira <gshap...@cloudera.com> >> wrote: >> >> > Take a look at ConsumerOffsetChecker. It does just that: print the >> > offset and lag for each consumer and partition. >> > >> > You can either use that class directly, or use it as a guideline for >> > your implementation >> > >> > On Wed, Oct 1, 2014 at 2:10 AM, Shlomi Hazan <shl...@viber.com> wrote: >> > > Hi, >> > > How can I programmatically get the number of items in a topic, pending >> > for >> > > consumption? >> > > If no programmatic way is avail, what other method is available? >> > > Shlomi >> > >> > >
