I have a topic with 16 partitions. I also have 24 consumer threads (8 per process per box) subscribed to that same topic. This configuration ensures that there is plenty of room for 1:1 partition to consumer assignment. And some standby consumers to take over in case the process dies
But during a kafka restart (one of many), I somehow ended up in a state where 16 partitions where assigned to only 12 consumer threads. ``` 6 partitions to 4 consumers on Box 1 5 partitions to 4 consumers on Box 2 5 partitions to 4 consumers on Box 3 ``` I believe at least some of the other consumers on each boxes were still active As a result, few of the consumers were overloaded resulting in a bigger lag. I initially thought this was due to kafka rebalancing but that doesn't seem to be the case. I am using the default RangeAssigner. I don't believe that anything will change with RoundRobinAssigner since I only have 1 topic. (Feel free to correct me if I am wrong about the round robin assigner) How can I ensure that partitions to consumer assignment is always uniform? Would appreciate if you have any insights. Thanks, Praveen
