Re: offset storage as kafka with zookeeper 3.4.6

2015-06-11 Thread Kris K
>If you want to "move" offsets from zookeeper to Kafka then yes you >need to have a phase where all consumers in your group set dual commit >to true. If you are starting a fresh consumer group then you can >turn off dual-commit. I followed these steps to move the offsets from zookeeper to kafka: 1.

Re: offset storage as kafka with zookeeper 3.4.6

2015-06-11 Thread Joel Koshy
> Is it mandatory to use the zookeeper that comes with kafka for offset > storage to be migrated to kafka? If you want to "move" offsets from zookeeper to Kafka then yes you need to have a phase where all consumers in your group set dual commit to true. If you are starting a fresh consumer group th

offset storage as kafka with zookeeper 3.4.6

2015-06-11 Thread Kris K
I am trying to migrate the offset storage to kafka (3 brokers of version 0.8.2.1) using the consumer property offsets.storage=kafka. I noticed that a new topic, __consumer_offsets got created. But nothing is being written to this topic, while the consumer offsets continue to reside on zookeeper.