Hi folks, Zookeeper is not required for storing offsets with the new consumer, but it still has a variety of other uses for Kafka and will always be required for coordination purposes. Please see Gwen's answer here as it's a pretty nice write up:
https://www.quora.com/What-is-the-actual-role-of-ZooKeeper-in-Kafka -Dustin On Tue, May 10, 2016 at 5:23 AM, Snehalata Nagaje < snehalata.nag...@harbingergroup.com> wrote: > > > But if we set autocommit to false and fetch data using simple consumer, > will it still use zookeeper for any purpose? > > > > ----- Original Message ----- > From: "Spico Florin" <spicoflo...@gmail.com> > To: users@kafka.apache.org > Sent: Tuesday, May 10, 2016 1:55:39 PM > Subject: Re: Kafka 9 version offset storage mechanism changes > > Hi! > Is just a guess (perhaps someone will correct me if I'm wrong). It > depends on the API you are using for consumers: > - simple API uses ZK for storring the offsets > - high level API stores the offests in Kafka Broker __consumer_offests > topic. > I hope it help. > Florin > > On Tue, May 10, 2016 at 11:17 AM, Mudit Kumar <mudit.ku...@askme.in> > wrote: > > > so zookeeper not needed anymore? > > > > > On May 10, 2016, at 1:46 PM, Spico Florin <spicoflo...@gmail.com> > wrote: > > > > > > Hi! > > > Yes both are possible. The new versions 0.9 and above store the offsets > > in > > > a special Kafka topic named __consumers_offsets. > > > Regards, > > > florin > > > > > > On Tue, May 10, 2016 at 8:33 AM, Gerard Klijs <gerard.kl...@dizzit.com > > > > > wrote: > > > > > >> Both are possible, but the 'new' consumer stores the offset in an > > __offset > > >> topic. > > >> > > >> On Tue, May 10, 2016 at 7:07 AM Snehalata Nagaje < > > >> snehalata.nag...@harbingergroup.com> wrote: > > >> > > >>> > > >>> > > >>> Hi All, > > >>> > > >>> > > >>> As per kafka 9 version, where does kafka store committed offset? > > >>> > > >>> is it in zookeeper or kafka broker? > > >>> > > >>> Also there is option to use offset storage outside kafka, does it > mean > > , > > >>> kafka will not depend on zookeepr for offset. > > >>> > > >>> Thanks, > > >>> snehalata > > >>> > > >> > > > > > -- Dustin Cote confluent.io
