Michael Bingham created KAFKA-9965:
--------------------------------------
Summary: Uneven distribution with RoundRobinPartitioner in AK 2.4+
Key: KAFKA-9965
URL: https://issues.apache.org/jira/browse/KAFKA-9965
Project: Kafka
Issue Type: Bug
Components: producer
Affects Versions: 2.4.1, 2.5.0, 2.4.0
Reporter: Michael Bingham
{{RoundRobinPartitioner}} states that it will provide equal distribution of
records across partitions. However with the enhancements made in KIP-480, it
may not. In some cases, when a new batch is started, the partitioner may be
called a second time for the same record:
[https://github.com/apache/kafka/blob/2.4/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L909]
[https://github.com/apache/kafka/blob/2.4/clients/src/main/java/org/apache/kafka/clients/producer/KafkaProducer.java#L934]
Each time the partitioner is called, it increments a counter in
{{RoundRobinPartitioner}}, so this can result in unequal distribution.
Easiest fix might be to decrement the counter in
{{RoundRobinPartitioner#onNewBatch}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
