Hello Martin, It seems your consumer's heartbeat.interval.ms config value is too small (default is 3 seconds) for your environment, consider increasing it and see if this issue goes away.
At the same time, we have some better error handling fixes in trunk which will be included in the next point release. https://issues.apache.org/jira/browse/KAFKA-2860 Guozhang On Wed, Nov 25, 2015 at 6:54 AM, Martin Skøtt < martin.sko...@falconsocial.com> wrote: > Hi, > > I'm experiencing some very strange issues with 0.9. I get these log > messages from the new consumer: > > [main] ERROR > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error > ILLEGAL_GENERATION occurred while committing offsets for group > aaa-bbb-reader > [main] WARN org.apache.kafka.clients.consumer.internals.ConsumerCoordinator > - Auto offset commit failed: Commit cannot be completed due to group > rebalance > [main] ERROR > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error > ILLEGAL_GENERATION occurred while committing offsets for group > aaa-bbb-reader > [main] WARN org.apache.kafka.clients.consumer.internals.ConsumerCoordinator > - Auto offset commit failed: > [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator > - Attempt to join group aaa-bbb-reader failed due to unknown member id, > resetting and retrying. > > And this in the broker log: > [2015-11-25 15:41:01,542] INFO [GroupCoordinator 0]: Preparing to > restabilize group aaa-bbb-reader with old generation 1 > (kafka.coordinator.GroupCoordinator) > [2015-11-25 15:41:01,544] INFO [GroupCoordinator 0]: > Group aaa-bbb-reader generation 1 is dead and removed > (kafka.coordinator.GroupCoordinator) > [2015-11-25 15:41:13,474] INFO [GroupCoordinator 0]: Preparing to > restabilize group aaa-bbb-reader with old generation 0 > (kafka.coordinator.GroupCoordinator) > [2015-11-25 15:41:13,475] INFO [GroupCoordinator 0]: Stabilized > group aaa-bbb-reader generation 1 (kafka.coordinator.GroupCoordinator) > [2015-11-25 15:41:13,477] INFO [GroupCoordinator 0]: Assignment received > from leader for group aaa-bbb-reader for generation 1 > (kafka.coordinator.GroupCoordinator) > [2015-11-25 15:41:43,478] INFO [GroupCoordinator 0]: Preparing to > restabilize group aaa-bbb-reader with old generation 1 > (kafka.coordinator.GroupCoordinator) > [2015-11-25 15:41:43,478] INFO [GroupCoordinator 0]: > Group aaa-bbb-reader generation 1 is dead and removed > (kafka.coordinator.GroupCoordinator) > > When this happens the kafka-consumer-groups describe command keeps saying > that the group no longer exists or is rebalancing. What is probably even > worse is that my consumers appears to be looping constantly through > everything written to the topics!? > > Does anyone have any input on what might be happening? > > I'm running 0.9 locally on my laptop using one Zookeeper and one broker, > both using the configuration provided in the distribution. I have 13 topics > with two partitions each and a replication factor of 1. I run one producer > and once consumer also on the same machine. > > -- > Martin Skøtt > -- -- Guozhang
