The topic deletion only triggers tombstone on brokers >= 0.10.2, correct? I
thought there was an outstanding bug report for this in lower versions...

On Wed, Feb 22, 2017 at 6:17 PM, Hans Jespersen <h...@confluent.io> wrote:

> The __consumer_offsets topic should also get a tombstone message as soon as
> a topic is deleted.
>
> -hans
>
> /**
>  * Hans Jespersen, Principal Systems Engineer, Confluent Inc.
>  * h...@confluent.io (650)924-2670
>  */
>
> On Wed, Feb 22, 2017 at 5:59 PM, Jun MA <mj.saber1...@gmail.com> wrote:
>
> > Hi Todd,
> >
> > Thank you so much for your reply. I assume that the broker will produce
> > the tombstone to __consumer_offsets topic when the offset expires? I’m
> > curious how broker notices the offset expires? Does it store the offset
> > message in memory and periodically check if someone expires?
> >
> > Thanks,
> > Jun
> >
> > > On Feb 22, 2017, at 4:37 PM, Todd Palino <tpal...@gmail.com> wrote:
> > >
> > > __consumer_offsets is a log-compacted topic, and a NULL body indicates
> a
> > > delete tombstone. So it means to delete the entry that matches the key
> > > (group, topic, partition tuple).
> > >
> > > -Todd
> > >
> > >
> > >
> > > On Wed, Feb 22, 2017 at 3:50 PM, Jun MA <mj.saber1...@gmail.com>
> wrote:
> > >
> > >> Hi guys,
> > >>
> > >> I’m trying to consume from __consumer_offsets topic to get exact
> > committed
> > >> offset of each consumer. Normally I can see messages like:
> > >>
> > >> [eds-els-recopp-jenkins-01-5651,eds-incre-staging-1,0]::[
> > >> OffsetMetadata[29791925,NO_METADATA],CommitTime
> > >> 1487090167367,ExpirationTime 1487176567367],
> > >>
> > >> which make sense to me. But sometimes I see messages like:
> > >>
> > >> [eds-elssearchindex-curiosity-stg-10892,eds-incre-v2-
> > staging-els,0]::NULL.
> > >>
> > >> Can someone explains what is NULL means here and why a NULL value get
> > >> published to __consumer_offsets?
> > >>
> > >> We’re running kafka 0.9.0.1 and we use org.apache.kafka.common.
> > >> serialization.ByteArrayDeserializer and GroupMetadataManager.
> > OffsetsMessageFormatter
> > >> to parse the message.
> > >>
> > >> Thanks,
> > >> Jun
> > >
> > >
> > >
> > >
> > > --
> > > *Todd Palino*
> > > Staff Site Reliability Engineer
> > > Data Infrastructure Streaming
> > >
> > >
> > >
> > > linkedin.com/in/toddpalino
> >
> >
>

Reply via email to