Hi Luke, I am committing the offsets manually, and after processing each of the records returned by one poll loop iteration using consumer.commitSync().
Regards, ________________________________ From: Luke Chen <show...@gmail.com> Sent: Monday, June 21, 2021 3:06 PM To: Kafka Users <users@kafka.apache.org> Subject: Re: Kafka incremental sticky rebalancing. Hi Mazen, Did you auto commit offsets or manually? When do you commit offsets? Thanks Luke Mazen Ezzeddine <mazen.ezzedd...@etu.univ-cotedazur.fr> 於 2021年6月21日 週一 下午7:42 寫道： > I am running Kafka on Kubernetes using the Kafka Strimzi operator. I am > using incremental sticky rebalance strategy by configuring my consumers > with the following: > > ConsumerConfig.PARTITION_ASSIGNMENT_STRATEGY_CONFIG, > > org.apache.kafka.clients.consumer.CooperativeStickyAssignor.class.getName() > > Each time I scale consumers in my consumer group all existing consumer in > the group generate the following exception > > Exception in thread "main" > org.apache.kafka.common.errors.RebalanceInProgressException: Offset commit > cannot be completed since the consumer is undergoing a rebalance for auto > partition assignment. You can try completing the rebalance by calling > poll() and then retry the operation > > > Any idea on what caused this exception and/or how to resolve it? > > > Thank you. > >
