Hi Todd, Thank you so much for your reply. I assume that the broker will produce the tombstone to __consumer_offsets topic when the offset expires? I’m curious how broker notices the offset expires? Does it store the offset message in memory and periodically check if someone expires?
Thanks, Jun > On Feb 22, 2017, at 4:37 PM, Todd Palino <tpal...@gmail.com> wrote: > > __consumer_offsets is a log-compacted topic, and a NULL body indicates a > delete tombstone. So it means to delete the entry that matches the key > (group, topic, partition tuple). > > -Todd > > > > On Wed, Feb 22, 2017 at 3:50 PM, Jun MA <mj.saber1...@gmail.com> wrote: > >> Hi guys, >> >> I’m trying to consume from __consumer_offsets topic to get exact committed >> offset of each consumer. Normally I can see messages like: >> >> [eds-els-recopp-jenkins-01-5651,eds-incre-staging-1,0]::[ >> OffsetMetadata[29791925,NO_METADATA],CommitTime >> 1487090167367,ExpirationTime 1487176567367], >> >> which make sense to me. But sometimes I see messages like: >> >> [eds-elssearchindex-curiosity-stg-10892,eds-incre-v2-staging-els,0]::NULL. >> >> Can someone explains what is NULL means here and why a NULL value get >> published to __consumer_offsets? >> >> We’re running kafka 0.9.0.1 and we use org.apache.kafka.common. >> serialization.ByteArrayDeserializer and >> GroupMetadataManager.OffsetsMessageFormatter >> to parse the message. >> >> Thanks, >> Jun > > > > > -- > *Todd Palino* > Staff Site Reliability Engineer > Data Infrastructure Streaming > > > > linkedin.com/in/toddpalino
