I'm trying to understand the config options for auto-rebalancing. This
is what we have in /etc/kafka/server.properties for all the nodes:
auto.leader.rebalance.enable=true
leader.imbalance.per.broker.percentage=10
leader.imbalance.check.interval.seconds=300
We have 10 nodes for this topic which has 512 partitions. They were
evenly balanced before I started my experiment. I shut down two of the
nodes, and the number of leaders per node is now:
75 10
68 3
57 4
67 5
57 6
68 7
63 8
57 9
Where the first column is # of leaders, and the second column is node #.
You can see that nodes 1 and 2 have no leaders, since they're down. It's
been about half an hour since I did this and the balancing hasn't changed.
The documentation on the config option is very ambiguous. My
interpretation is that it says if any particular node has 10% more
leaders then auto-rebalance kicks in. If that means 10% more than the
average, then node #10 has 75 partitioners, and the average is 64, so
that's a 17% difference.
So I think I'm misunderstanding either what auto-rebalance is supposed
to do or the condition that's supposed to trigger it. Any clues?
Thanks,
Wes
