Hello Martin, Could you paste the consumer config values in this thread as well? And is the consumer co-located with the broker?
Guozhang On Wed, Nov 18, 2015 at 7:40 AM, Martin Skøtt < martin.sko...@falconsocial.com> wrote: > Hi, > > I'm tracking the 0.9.0.0 Git tag and have a Java consumer using the new > API, but I'm seeing some strange issues. I run ZooKeeper and Kafka on my > own machine using the settings files in config/ and no authentication. > Build is done using Oracle JDK 8. I have 13 topics, each created with a > replication factor of 1. > > The first time my consumer connects to the newly created topics (before > anything has been written to the topic) it emits the following errors: > > [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka version : > 0.9.0.0 > [main] INFO org.apache.kafka.common.utils.AppInfoParser - Kafka commitId : > fe855f98263cb7b8 > [main] ERROR > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto > offset commit failed. > org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException: The > group coordinator is not available. > [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator > - Marking the coordinator 2147483647 dead. > [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator > - Attempt to join group xxx-yyy-reader failed due to obsolete coordinator > information, retrying. > [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator > - Marking the coordinator 2147483647 dead. > [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator > - Attempt to join group xxx-yyy-reader failed due to obsolete coordinator > information, retrying. > > If I restart my consumer then it starts without issue. > > When I start writing to the topics then my consumer reads data, but will > often log messages like these: > > [main] ERROR > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto > offset commit failed. > org.apache.kafka.clients.consumer.internals.SendFailedException > > And occasionally these: > [main] INFO org.apache.kafka.clients.consumer.internals.AbstractCoordinator > - Attempt to heart beat failed since member id is not valid, reset it and > try to re-join group. > [main] ERROR > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Error > ILLEGAL_GENERATION occurred while committing offsets for group > audience-indexer-reader > [main] ERROR > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Auto > offset commit failed. > org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be > completed due to group rebalance > at > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:497) > at > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:438) > at > > org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:673) > at > > org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:646) > at > > org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:167) > at > > org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:133) > at > > org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:107) > at > > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.onComplete(ConsumerNetworkClient.java:350) > at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:288) > at > > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.clientPoll(ConsumerNetworkClient.java:303) > at > > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:197) > at > > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:187) > at > > org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:157) > at > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:352) > at > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.maybeAutoCommitOffsetsSync(ConsumerCoordinator.java:384) > at > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinPrepare(ConsumerCoordinator.java:236) > at > > org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:209) > at > > org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.ensurePartitionAssignment(ConsumerCoordinator.java:304) > at > > org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:861) > at > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:829) > > Does anyone have any input on what might be happening? > > -- > Martin Skøtt > -- -- Guozhang
