Hi Han, if it doesn't work you should file an issue, since it explicitly says in the readme that it works with: 1 zookeeper built-in high-level consumer (based on Zookeeper) 2 kafka built-in offset management API (based on Kafka internal topic) 3 Storm Kafka Spout (based on Zookeeper by default)
I'm still on Kafka 0.8, so I can't shed any light on your issue. Thx for the AdminClient info. Marko Bonaći Monitoring | Alerting | Anomaly Detection | Centralized Log Management Solr & Elasticsearch Support Sematext <http://sematext.com/> | Contact <http://sematext.com/about/contact.html> On Wed, Dec 30, 2015 at 6:10 PM, Han JU <ju.han.fe...@gmail.com> wrote: > Hi Marko, > > Yes we're currently using this on our production kafka 0.8. But it does not > seem to work with the new consumer API in 0.9. > To answer my own question about deleting consumer group in new consumer > API, it seems that it's currently not possible with the new consumer API > (there's no delete related method in `AdminClient` of the new consumer > API). > > > 2015-12-30 17:02 GMT+01:00 Marko Bonaći <marko.bon...@sematext.com>: > > > If you want to monitor offset (ZK or Kafka based), try with QuantFind's > > Kafka Offset Monitor. > > If you use Docker, it's easy as: > > > > docker run -p 8080:8080 -e ZK=zk_hostname:2181 > > jpodeszwik/kafka-offset-monitor > > and then opening browser to dockerhost:8080. > > > > If not in the Docker mood, use instructions here: > > https://github.com/quantifind/KafkaOffsetMonitor > > > > Marko Bonaći > > Monitoring | Alerting | Anomaly Detection | Centralized Log Management > > Solr & Elasticsearch Support > > Sematext <http://sematext.com/> | Contact > > <http://sematext.com/about/contact.html> > > > > On Wed, Dec 30, 2015 at 12:54 PM, Han JU <ju.han.fe...@gmail.com> wrote: > > > > > Thanks guys. The `seek` seems a solution. But it's more cumbersome than > > in > > > 0.8 because I have to plug in some extra code in my consumer > abstractions > > > rather than simply deleting a zk node. > > > And one more question: where does kafka 0.9 stores the consumer-group > > > information? In fact I also tried to delete the consumer group but the > > > `AdminUtils.deleteConsumerGroupInZK` does not seem to work in 0.9. And > > also > > > `bin/kafka-run-class.sh kafka.tools.ConsumerOffsetChecker --zookeeper > > > localhost:2181 --group group-name` seems broken. > > > > > > Thanks! > > > > > > 2015-12-29 16:46 GMT+01:00 Marko Bonaći <marko.bon...@sematext.com>: > > > > > > > I was refering to Dana Powers's answer in the link I posted (to use a > > > > client API). You can find an example here: > > > > > > > > > > > > > > http://kafka.apache.org/090/javadoc/org/apache/kafka/clients/consumer/KafkaConsumer.html > > > > > > > > Marko Bonaći > > > > Monitoring | Alerting | Anomaly Detection | Centralized Log > Management > > > > Solr & Elasticsearch Support > > > > Sematext <http://sematext.com/> | Contact > > > > <http://sematext.com/about/contact.html> > > > > > > > > On Tue, Dec 29, 2015 at 4:41 PM, Stevo Slavić <ssla...@gmail.com> > > wrote: > > > > > > > > > Then I guess @Before test, explicitly commit offset of 0. > > > > > > > > > > There doesn't seem to be a tool for committing offset, only for > > > > > checking/fetching current offset (see > > > > > http://kafka.apache.org/documentation.html#operations ) > > > > > > > > > > On Tue, Dec 29, 2015 at 4:35 PM, Han JU <ju.han.fe...@gmail.com> > > > wrote: > > > > > > > > > > > Hi Stevo, > > > > > > > > > > > > But by deleting and recreating the topic, do I remove also the > > > messages > > > > > > ingested? > > > > > > My use case is that I ingest prepared messages once and run > > consumer > > > > > tests > > > > > > multiple times, between each test run I reset the consumer > group's > > > > offset > > > > > > so that each run starts from the beginning and consumers all the > > > > > messages. > > > > > > > > > > > > 2015-12-29 16:19 GMT+01:00 Stevo Slavić <ssla...@gmail.com>: > > > > > > > > > > > > > Have you considered deleting and recreating topic used in test? > > > > > > > Once topic is clean, read/poll once - any committed offset > should > > > be > > > > > > > outside of the range, and consumer should reset offset. > > > > > > > > > > > > > > On Tue, Dec 29, 2015 at 4:11 PM, Han JU < > ju.han.fe...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > For local test purpose I need to frequently reset offset for > a > > > > > consumer > > > > > > > > group. In 0.8 I just delete the consumer group's zk node > under > > > > > > > > `/consumers`. But with the redesign of the 0.9, how could I > > > achieve > > > > > the > > > > > > > > same thing? > > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > -- > > > > > > > > *JU Han* > > > > > > > > > > > > > > > > Software Engineer @ Teads.tv > > > > > > > > > > > > > > > > +33 0619608888 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > *JU Han* > > > > > > > > > > > > Software Engineer @ Teads.tv > > > > > > > > > > > > +33 0619608888 > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > *JU Han* > > > > > > Software Engineer @ Teads.tv > > > > > > +33 0619608888 > > > > > > > > > -- > *JU Han* > > Software Engineer @ Teads.tv > > +33 0619608888 >
