Re: Question about messages in __consumer_offsets topic

2017-02-23 Thread James Cheng
Yup, this got fixed in 0.10.2 https://issues.apache.org/jira/browse/KAFKA-2000 -James > On Feb 23, 2017, at 11:10 AM, Jeff Widman wrote: > > The topic deletion only triggers tombstone on brokers >= 0.10.2, correct? I > thought there was an ou

Re: Question about messages in __consumer_offsets topic

2017-02-23 Thread Jeff Widman
The topic deletion only triggers tombstone on brokers >= 0.10.2, correct? I thought there was an outstanding bug report for this in lower versions... On Wed, Feb 22, 2017 at 6:17 PM, Hans Jespersen wrote: > The __consumer_offsets topic should also get a tombstone message as soon as > a topic is

Re: Question about messages in __consumer_offsets topic

2017-02-22 Thread Mathieu Fenniak
Hi Jun, I ran into the same question today (see thread, subject: Consumer / Streams causes deletes in __consumer_offsets?), and here's what Eno and Guozhang helped me understand: There are broker-level configuration values called "offsets.retention.minutes" and "offsets.retention.check.interval.m

Re: Question about messages in __consumer_offsets topic

2017-02-22 Thread Hans Jespersen
The __consumer_offsets topic should also get a tombstone message as soon as a topic is deleted. -hans /** * Hans Jespersen, Principal Systems Engineer, Confluent Inc. * h...@confluent.io (650)924-2670 */ On Wed, Feb 22, 2017 at 5:59 PM, Jun MA wrote: > Hi Todd, > > Thank you so much for you

Re: Question about messages in __consumer_offsets topic

2017-02-22 Thread Jun MA
Hi Todd, Thank you so much for your reply. I assume that the broker will produce the tombstone to __consumer_offsets topic when the offset expires? I’m curious how broker notices the offset expires? Does it store the offset message in memory and periodically check if someone expires? Thanks, J

Re: Question about messages in __consumer_offsets topic

2017-02-22 Thread Todd Palino
__consumer_offsets is a log-compacted topic, and a NULL body indicates a delete tombstone. So it means to delete the entry that matches the key (group, topic, partition tuple). -Todd On Wed, Feb 22, 2017 at 3:50 PM, Jun MA wrote: > Hi guys, > > I’m trying to consume from __consumer_offsets to