Re: Kafka rebalancing - assignement of Kafka consumers to partitions

2020-10-12 Thread Shaohan Yin
Hi, If you are using the official Java clients: (1) Partition assignments are performed on the client side. You could check the interface named ConsumerPartitionAssignor or PartitionAssignor ( deprecated in 2.4 ) for customized partition assignments. (2) You could implement the ConsumerPartitionA

Kafka rebalancing - assignement of Kafka consumers to partitions

2020-10-12 Thread Mazen Ezzeddine
When a new Kafka consumer joins/leaves a consumer group, Kafka runtime triggers a rebalancing process so that a new assignment/mapping of partitions to the new set of consumers is performed. I kindly have three questions on the rebalancing process: (1) Is it possible to plug in somehow a custom