Re: Coordinator lost for consumer groups

You might try increasing the log.cleaner.dedupe.buffer.size. This should increase the deduplication yield for each scan. If you haven’t seen them there are some notes on log compaction here: https://cwiki.apache.org/confluence/display/KAFKA/Log+Compaction

Re: Coordinator lost for consumer groups

The problem still persists. But note that I was running old consumer (Zk based) to describe consumers. Running ./kafka-consumer-groups.sh kafka-groups.sh --bootstrap-server 10.211.16.215 --group groupX --describe, I get the below error. So none of the consumer groups seem to have a coordinator now.

Re: Coordinator lost for consumer groups

And I am willing to suspend log compaction and restart the brokers, but am worried if that will leave the system in a recoverable state or If I just have to wait it out. On Fri, Jul 1, 2016 at 2:06 PM, Sathyakumar Seshachalam < sathyakumar_seshacha...@trimble.com> wrote: > Hi, > > I have 3 Kafka

Coordinator lost for consumer groups

Hi, I have 3 Kafka nodes (running 0.9.0) that all had active consumers and producers. Now all these had uncompacted __consumer_offsets group that grew to 1.8 TB. So I restarted these nodes with a log.cleaner.enabled to save some space. Since then consumers have stalled. When I do a ./kafka-consu