Re: Auto Purging Consumer Group Configuration [Especially Kafka Console Group]

There is a JIRA open for exactly this issue with patch available: KAFKA-559 On Fri, Oct 10, 2014 at 9:33 AM, Neha Narkhede wrote: > Bhavesh, > > All consumer groups with at least one alive consumer will have a > registration node for that consume

Re: Auto Purging Consumer Group Configuration [Especially Kafka Console Group]

Bhavesh, All consumer groups with at least one alive consumer will have a registration node for that consumer instance under /consumers//ids. Any consumer group that is long gone will have no registered consumer instance in zookeeper. The right way to clean up defunct consumer groups is to scan co

Re: Auto Purging Consumer Group Configuration [Especially Kafka Console Group]

We just want to clean-up old configuration from ZK. We can check from the offset API so we can delete based on offset .. is that right ? there is no date last associated with Consumer Group ? Is that right in ZK configuration ? Thanks, Bhavesh On Thu, Oct 9, 2014 at 9:23 PM, Gwen Shapira wr

Re: Auto Purging Consumer Group Configuration [Especially Kafka Console Group]

The problem with Kafka is that we never know when a consumer is "truly" inactive. But - if you decide to define inactive as consumer who's last offset is lower than anything available on the log (or perhaps lagging by over X messages?), its fairly easy to write a script to detect and clean them di

Auto Purging Consumer Group Configuration [Especially Kafka Console Group]

Hi Kafka, We have lots of lingering console consumer group people have created for testing or debugging purpose for one time use via bin/kafka-console-consumer.sh. Is there auto purging that clean script that Kafka provide ? Is three any API to find out inactive Consumer group and delete consume