Thanks for this KIP (and the companion PR) This is an interesting proposal. I have to admit that I've only used this tool when I want to force a specific topic/ReplicaSet distribution.
I wonder what criteria you used when deciding the set of goals and their order. Reason I ask is because we use MirrorMaker extensively to balance our clusters, and not only it offers lots of different goals, but also allows operators to decide which ones to optimize for. My worry is that having a fixed/opinionated criteria on how replicas are distributed can be limiting. From: dev@kafka.apache.org At: 03/28/25 23:04:50 UTC-4:00To: dev@kafka.apache.org Subject: [DISCUSS] KIP-1151: Minimal Movement Replica Balancing Algorithm for Reassignment Hi Kafka Community, I would like to start a discussion on a new KIP proposal: KIP-1151: Minimal movement replica balancing algorithm for reassignment Motivation Kafka clusters require rebalancing of topic replicas after horizontal scaling to evenly distribute the load. However, the current approach does not consider the existing replica distribution, often leading to unnecessary movements. This KIP proposes an optimized strategy that minimizes movements while achieving an even distribution. Proposal The proposal focuses on: 1. Minimal Movement: Minimize the number of replica relocations during rebalancing. 2. Replica Balancing: Ensure that replicas are evenly distributed across brokers. 3. Anti-Affinity Support: Support rack-aware allocation when enabled. 4. Leader Balancing: Distribute leader replicas evenly across brokers. 5.ISR Order Optimization: Optimize adjacency relationships to prevent failover traffic concentration in case of broker failures. For a detailed description, please refer to the KIP document: KIP-1151: https://cwiki.apache.org/confluence/display/KAFKA/KIP-1151%3A+Minimal+movement+r eplica+balancing+algorithm+for+reassignment I would appreciate any feedback, comments, or suggestions from the community. Best
