Ah, yes those consumers znodes in Zookeeper are related to consumer group
metadata.  You can actually find a full write up of what each znode is used
for here:
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper

Cheers,

On Wed, May 11, 2016 at 9:22 AM, Spico Florin <spicoflo...@gmail.com> wrote:

> Hi, Dustin!
>  Thank you for your answer. I have observed that in zookeeper there was a
> folder named consumers, that kept data about the topic name its partitions
> and the offsets.
> Were both the consumers/producers using this folder to keep track of the
> offsets? What was the purpose of this folder?
> I look forward for your answers.
>  Regards,
>  Florin
>
> On Wed, May 11, 2016 at 4:12 PM, Dustin Cote <dus...@confluent.io> wrote:
>
> > Hi Florin,
> >
> > The new consumer is intended to replace both the high level and simple
> > consumers.
> >
> > http://kafka.apache.org/documentation.html#consumerapi
> >
> > The old simple consumer API didn't rely on zookeeper to store offsets,
> but
> > rather the client was responsible for managing their own offsets as there
> > was no consumer group associated to a simple consumer.  However, your
> > simple consumer did talk to zookeeper to find out about partition leaders
> > and there is some handling required there.
> >
> >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+SimpleConsumer+Example
> >
> > I think Gwen's answer is complete in that respect.  Hope that helps.
> >
> > On Wed, May 11, 2016 at 3:23 AM, Spico Florin <spicoflo...@gmail.com>
> > wrote:
> >
> > > Hi!
> > > In Gwen's answer there is nothing related with the simple API consumer
> > and
> > > the relation with ZK. But still the simple consumer API uses ZK to
> store
> > > offsets, isn't it?
> > > I look forward for your answer.
> > >  Regards,
> > >  Florin
> > >
> > > On Wed, May 11, 2016 at 2:11 AM, R Krishna <krishna...@gmail.com>
> wrote:
> > >
> > > > And where is the documentation for this topic: "__consumers_offsets"
> > > >
> > > > On Tue, May 10, 2016 at 1:16 AM, Spico Florin <spicoflo...@gmail.com
> >
> > > > wrote:
> > > >
> > > > > Hi!
> > > > >  Yes both are possible. The new versions 0.9 and above store the
> > > offsets
> > > > in
> > > > > a special Kafka topic named __consumers_offsets.
> > > > > Regards,
> > > > >  florin
> > > > >
> > > > > On Tue, May 10, 2016 at 8:33 AM, Gerard Klijs <
> > gerard.kl...@dizzit.com
> > > >
> > > > > wrote:
> > > > >
> > > > > > Both are possible, but the 'new' consumer stores the offset in an
> > > > > __offset
> > > > > > topic.
> > > > > >
> > > > > > On Tue, May 10, 2016 at 7:07 AM Snehalata Nagaje <
> > > > > > snehalata.nag...@harbingergroup.com> wrote:
> > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Hi All,
> > > > > > >
> > > > > > >
> > > > > > > As per kafka 9 version, where does kafka store committed
> offset?
> > > > > > >
> > > > > > > is it in zookeeper or kafka broker?
> > > > > > >
> > > > > > > Also there is option to use offset storage outside kafka, does
> it
> > > > mean
> > > > > ,
> > > > > > > kafka will not depend on zookeepr for offset.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > snehalata
> > > > > > >
> > > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Radha Krishna, Proddaturi
> > > > 253-234-5657
> > > >
> > >
> >
> >
> >
> > --
> > Dustin Cote
> > confluent.io
> >
>



-- 
Dustin Cote
confluent.io

Reply via email to