You're probably spending too long processing some messages. The ILLEGAL_GENERATION message indicates that a new generation was created, which probably means you didn't call poll() frequently enough to send heartbeats indicating you are still an active member of the group.
We're currently working on implementing KIP-62 ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-62%3A+Allow+consumer+to+send+heartbeats+from+a+background+thread) to support background heartbeats during some timeout period so you can spend more than the session timeout processing data without causing you to fall out of the group. This helps decouple the amount of time spent processing messages from the timeout for tracking when a consumer has died and won't return. In the mean time, if you can, you might want to set max.poll.records to a small value to avoid hitting these timeouts. -Ewen On Tue, Jul 12, 2016 at 10:35 AM, Paramesh Nc <parameswa...@gmail.com> wrote: > Hi , > > I am getting the below error messages when consuming the records from the > kafka topic . > > o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION > occurred while committing offsets for group test-group. > > 2016-07-11 22:56:50.314 ERROR 8699 --- [ Thread] > o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION > occurred while committing offsets for group test-group > 2016-07-11 22:56:50.315 WARN 8699 --- [Thread] > o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed: > Commit cannot be completed due to group rebalance > 2016-07-11 22:56:50.320 ERROR 8699 --- [ Thread] > o.a.k.c.c.internals.ConsumerCoordinator : Error ILLEGAL_GENERATION > occurred while committing offsets for group test-group > 2016-07-11 22:56:50.321 WARN 8699 --- [- Thread] > o.a.k.c.c.internals.ConsumerCoordinator : Auto offset commit failed: > 2016-07-11 22:56:50.323 INFO 8699 --- [Thread] > o.a.k.c.c.internals.AbstractCoordinator : Attempt to join group test-group > failed due to unknown member id, resetting and retrying. > > And I am creating the consumer as follows : > KafkaConsumer<byte[],byte[]> consumer=new KafkaConsumer<>(props); > consumer.subscribe(Arrays.asList(topic). > Where props is set with the properties : > props.put("security.protocol", "SASL_PLAINTEXT"); > props.put("group.id", groupId); > props.put("bootstrap.servers", "pqrs:9092,xyz:9092"); > And kafka client I am using is 0.9.0.1. > > > > Could you please help me to resolve this issue. I have gone through so many > links available over the internet but did not find the useful answers. > > Thanks in Advance. > Param. > -- Thanks, Ewen
