Mathias Herberts created KAFKA-4040:
---------------------------------------
Summary: Round Robin Assignment does not create balanced
assignments
Key: KAFKA-4040
URL: https://issues.apache.org/jira/browse/KAFKA-4040
Project: Kafka
Issue Type: Improvement
Affects Versions: 0.8.2.2
Reporter: Mathias Herberts
The RoundRobinAssignor loops over the consumers in a circular manner, assigning
each time N partitions to each consumer, where N is the number of consuming
threads.
In the following scenario, this creates a major imbalance in the assignment.
single topic with 64 partitions, 4 consumers (A,B,C,D), each with 12 threads.
The roundrobin strategy will allocate the partitions in the following manner:
A: 24
B: 16
C: 12
D: 12
when the expected assignment would rather be 16 to each consumer.
The reason for this imbalance is that instead of allocating a single partition
to each consumer in sequence, the assignor attempts to assign 12 partitions
each time it is handed a consumer by the circularIterator, so it starts by
assigning 12 to each of A,B,C and D, then it has 16 partitions left and it is
handed A again, to which it assigns 12 partitions before moving to B to which 4
will be assigned.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
