Getting consumers list from Kafka Metadata API

2014-11-30 Thread Mo Firouz
Hi guys, I was wondering if I could get a list of consumers / consumer groups from Kafka directly without the use of Zookeeper? Bonus point goes if I can get a list of producers too? I'm currently working on a little monitoring tool for Kafka 0.8 ( https://github.com/mofirouz/kafka-monitor). Hop

Re: Kafka Migration Tool

2014-05-21 Thread Mo Firouz
Thanks, you are right, my consumer config was pointing to the 0.8! Cheers! On 20 May 2014 05:36, Jun Rao wrote: > It seems that you may have set the zk connection string to one used by 0.8 > Kafka brokers. > > Thanks, > > Jun > > > On Mon, May 19, 2014 at 8:34 AM, M

Kafka Migration Tool

2014-05-19 Thread Mo Firouz
Hi all, I'm trying to migrate from Kafka 0.7.2-2.9.2 (with Zookeeper 3.3.4 from Cloudera) to Kafka 0.8.1.1-2.9.2 (with official Zookeeper 3.4.5 ) - However hitting brick walls with a very mysterious problem: 6) at kafka.tools.KafkaMigrationTool.main(KafkaMigrationTool.java:217) Caused by: java.la

Re: Migration Tool consumer rebalance error when attempting to migrate from Kafka 0.7 to Kafka 0.8

2014-03-20 Thread Mo Firouz
Hi Jun, Thanks for your response and help. Yes indeed BOTH the kafka and zookeeper variations were different that was causing this error! It's now working! Thanks for you help and time! Cheers, Mo. On 19 March 2014 14:07, Mo Firouz wrote: > Hello. > > I am trying to migrate

Re: Migration Tool consumer rebalance error when attempting to migrate from Kafka 0.7 to Kafka 0.8

2014-03-19 Thread Mo Firouz
avaapi.consumer.ZookeeperConsumerConnector.createMessageStreamsByFilter(ZookeeperConsumerConnector.scala:100) ... 5 more [2014-03-19 17:35:40,783] DEBUG Got ping response for sessionid: 0x3448da1f8f0644b after 7ms (org.apache.zookeeper.ClientCnxn) [2014-03-19 17:35:42,786] DEBUG Got ping response

Migration Tool consumer rebalance error when attempting to migrate from Kafka 0.7 to Kafka 0.8

2014-03-19 Thread Mo Firouz
Hello. I am trying to migrate from Kafka 0.7 to 0.8, and the first stage of this is to use the Migration Tool. I have setup a local Kafka 0.8 cluster consisting of 3 nodes on my local Vagrant (along with a zookeeper running locally). The Migration Tool Producer will connect to these three nodes i