The topic deletion only triggers tombstone on brokers >= 0.10.2, correct? I thought there was an outstanding bug report for this in lower versions...
On Wed, Feb 22, 2017 at 6:17 PM, Hans Jespersen <h...@confluent.io> wrote: > The __consumer_offsets topic should also get a tombstone message as soon as > a topic is deleted. > > -hans > > /** > * Hans Jespersen, Principal Systems Engineer, Confluent Inc. > * h...@confluent.io (650)924-2670 > */ > > On Wed, Feb 22, 2017 at 5:59 PM, Jun MA <mj.saber1...@gmail.com> wrote: > > > Hi Todd, > > > > Thank you so much for your reply. I assume that the broker will produce > > the tombstone to __consumer_offsets topic when the offset expires? I’m > > curious how broker notices the offset expires? Does it store the offset > > message in memory and periodically check if someone expires? > > > > Thanks, > > Jun > > > > > On Feb 22, 2017, at 4:37 PM, Todd Palino <tpal...@gmail.com> wrote: > > > > > > __consumer_offsets is a log-compacted topic, and a NULL body indicates > a > > > delete tombstone. So it means to delete the entry that matches the key > > > (group, topic, partition tuple). > > > > > > -Todd > > > > > > > > > > > > On Wed, Feb 22, 2017 at 3:50 PM, Jun MA <mj.saber1...@gmail.com> > wrote: > > > > > >> Hi guys, > > >> > > >> I’m trying to consume from __consumer_offsets topic to get exact > > committed > > >> offset of each consumer. Normally I can see messages like: > > >> > > >> [eds-els-recopp-jenkins-01-5651,eds-incre-staging-1,0]::[ > > >> OffsetMetadata[29791925,NO_METADATA],CommitTime > > >> 1487090167367,ExpirationTime 1487176567367], > > >> > > >> which make sense to me. But sometimes I see messages like: > > >> > > >> [eds-elssearchindex-curiosity-stg-10892,eds-incre-v2- > > staging-els,0]::NULL. > > >> > > >> Can someone explains what is NULL means here and why a NULL value get > > >> published to __consumer_offsets? > > >> > > >> We’re running kafka 0.9.0.1 and we use org.apache.kafka.common. > > >> serialization.ByteArrayDeserializer and GroupMetadataManager. > > OffsetsMessageFormatter > > >> to parse the message. > > >> > > >> Thanks, > > >> Jun > > > > > > > > > > > > > > > -- > > > *Todd Palino* > > > Staff Site Reliability Engineer > > > Data Infrastructure Streaming > > > > > > > > > > > > linkedin.com/in/toddpalino > > > > >
