Please note that recently I was working on tasks other than incremental rebalancing, but now I have returned to experimenting with incremental rebalancing.
Please note that I am running kafka broker clients version 2.7 on Kubernetes GKE. My setup consist of dynamic autoscaling of consumers configured with the CooperativeStickyAssignor As recently discussed, when a scale up is performed and a new consumer joins the group the below exception is raised. "Exception in thread "main" org.apache.kafka.common.errors.RebalanceInProgressException: Offset commit cannot be completed since the consumer is undergoing a rebalance for auto partition assignment. You can try completing the rebalance by calling poll() and then retry the operation." You suggested to just ignore the exception as the offset commit can be completed in the next poll round. Please note that as I scale the consumer group, all the members of the group are raising this exception. Given that I am running on Kubernetes, the consumer pods are considered in error state and Killed/restarted by Kubernetes, causing additional delays in addition to rebalancing delays (the failed one due to the exception raised, and the one caused by restarting). I don’t think this is normal? Anyway, could you please give me a hint on why this is happening, can it be because the rebalancing is taking place before the offset commit request has reached the coordinator (but this is a blocking commit so can not happen…) So please give me a hint on why this is happening and how to correct this weird behavior/exception. Thank you. ________________________________ From: Luke Chen <show...@gmail.com> Sent: Tuesday, June 22, 2021 3:58 AM To: Kafka Users <users@kafka.apache.org> Subject: Re: Kafka incremental sticky rebalancing. Hi Mazen, I think you can ignore this message as the message said, you can try completing the rebalance by calling poll() and then retry the operation. You should be able to complete the offset commit in the next poll round. Thanks. Luke On Mon, Jun 21, 2021 at 10:19 PM Mazen Ezzeddine < mazen.ezzedd...@etu.univ-cotedazur.fr> wrote: > Hi Luke, > > I am committing the offsets manually, and after processing each of the > records returned by one poll loop iteration using consumer.commitSync(). > > Regards, > > ________________________________ > From: Luke Chen <show...@gmail.com> > Sent: Monday, June 21, 2021 3:06 PM > To: Kafka Users <users@kafka.apache.org> > Subject: Re: Kafka incremental sticky rebalancing. > > Hi Mazen, > Did you auto commit offsets or manually? When do you commit offsets? > > Thanks > Luke > > Mazen Ezzeddine <mazen.ezzedd...@etu.univ-cotedazur.fr> 於 2021年6月21日 週一 > 下午7:42 寫道： > > > I am running Kafka on Kubernetes using the Kafka Strimzi operator. I am > > using incremental sticky rebalance strategy by configuring my consumers > > with the following: > > > > ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG, > > > > > org.apache.kafka.clients.consumer.CooperativeStickyAssignor.class.getName() > > > > Each time I scale consumers in my consumer group all existing consumer in > > the group generate the following exception > > > > Exception in thread "main" > > org.apache.kafka.common.errors.RebalanceInProgressException: Offset > commit > > cannot be completed since the consumer is undergoing a rebalance for auto > > partition assignment. You can try completing the rebalance by calling > > poll() and then retry the operation > > > > > > Any idea on what caused this exception and/or how to resolve it? > > > > > > Thank you. > > > > >
