Yeah, the current ConsumerOffsetChecker has this issue (maybe bug also) if the offset storage is Kafka and no offset has been committed. It will throw ZK exception, which is very confusing. KAFKA-1951 was opened for this but was not checked in.
Thanks. Jiangjie (Becket) Qin On 4/13/15, 9:55 AM, "4mayank" <4may...@gmail.com> wrote: >I did a similar change - moved from High Level Consumer to Simple >Consumer. >Howerver kafka-consumer-offset-checker.sh throws an exception. Its >searching the zk path /consumers/<group>/ which does not exist on any of >my >zk nodes. > >Is there any other tool for getting the offset lag when using Simple >Consumer? Or am I using kafka-consumer-offset-checker.sh incorrectly for >Simple Consumer? > >Output: >kafka-consumer-offset-checker.sh --zookeeper 192.168.1.201:2181, >192.168.1.202:2181,192.168.1.203:2181 --group my-control-group >Exiting due to: org.apache.zookeeper.KeeperException$NoNodeException: >KeeperErrorCode = NoNode for /consumers/my-control-group/owners. > > >kafka-consumer-offset-checker.sh --zookeeper 192.168.1.201:2181, >192.168.1.202:2181,192.168.1.203:2181 --group my-control-group --topic >my-control >Exiting due to: org.apache.zookeeper.KeeperException$NoNodeException: >KeeperErrorCode = NoNode for >/consumers/my-control-group/offsets/my-control/1. > > >ZK cli output: > >./zookeeper-shell.sh 192.168.1.201:2181 >Connecting to 192.168.1.201:2181 >Welcome to ZooKeeper! >JLine support is disabled > >WATCHER:: > >WatchedEvent state:SyncConnected type:None path:null >ls /config/topics >[my-control] >ls /consumers >[] >quit > > >Thanks. >Mayank. > >On Fri, Mar 20, 2015 at 9:54 AM, Jiangjie Qin <j...@linkedin.com.invalid> >wrote: > >> Hi Vamsi, >> >> The ConsumerOffsetChecker.scala or kafka-consumer-offset-checker.sh >>still >> works. You can use them to check the offsets. >> If you need to check the offsets programmatically, you can send >> OffsetsFetcheRequest to broker using simple consumer. You may refer the >> ConsumerOffsetChecker.scala to see how to find correct broker where >> corresponding offset manager resides. >> >> Jiangjie (Becket) Qin >> >> On 3/19/15, 11:54 PM, "Achanta Vamsi Subhash" >><achanta.va...@flipkart.com> >> wrote: >> >> >Hi, >> > >> >We are using 0.8.2.1 currently. >> > >> >- How to get the consumer offsets from the offsets topic? >> >​- Is there any built-in function which I could use? (like in >> >AdminUtils.scala) >> >- Is it ok to start a simple consumer and read the offsets from the >>topic? >> > >> >​We used to read the offsets from zookeeper previously for the >> >HighLevelConsumers. But with the new broker this changed as we are >>using >> >Kafka topic for offsets.​ >> > >> >-- >> >Regards >> >Vamsi Subhash >> >>
