Re: Strictly one rebalancing when scaling a consumer group to a Random number on kubernetes.

2021-03-29 Thread Mazen Ezzeddine
. From: Liam Clarke-Hutchinson Sent: Monday, March 29, 2021 10:51 AM To: users@kafka.apache.org Subject: Re: Strictly one rebalancing when scaling a consumer group to a Random number on kubernetes. Hi Mazzen, Have you looked into incremental cooperative rebalancing? It may help with your

Re: Strictly one rebalancing when scaling a consumer group to a Random number on kubernetes.

2021-03-29 Thread Liam Clarke-Hutchinson
Hi Mazzen, Have you looked into incremental cooperative rebalancing? It may help with your issues, at least it can avoid stop the world rebalances. https://www.confluent.io/blog/incremental-cooperative-rebalancing-in-kafka/ Cheers, Liam Clarke On Mon, 29 Mar. 2021, 8:04 pm Mazen Ezzeddine, < m

Strictly one rebalancing when scaling a consumer group to a Random number on kubernetes.

2021-03-29 Thread Mazen Ezzeddine
Hi all, Given a replicaset/statefulset of kafka consumers that constitute a consumer group running on kubernetes, if the number of replicas is x than sometimes x rebalancing might be triggered since not all of the replicas/consumers send a join group request in a timely and synced manner to the