ableegoldman commented on a change in pull request #8668:
URL: https://github.com/apache/kafka/pull/8668#discussion_r432802206
##########
File path:
clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractStickyAssignor.java
##########
@@ -65,9 +72,206 @@ public MemberData(List<TopicPartition> partitions,
Optional<Integer> generation)
@Override
public Map<String, List<TopicPartition>> assign(Map<String, Integer>
partitionsPerTopic,
Map<String, Subscription>
subscriptions) {
+ Map<String, List<TopicPartition>> consumerToOwnedPartitions = new
HashMap<>();
+ if (allSubscriptionsEqual(partitionsPerTopic.keySet(), subscriptions,
consumerToOwnedPartitions)) {
+ log.debug("Detected that all consumers were subscribed to same set
of topics, invoking the "
+ + "optimized assignment algorithm");
+ partitionsTransferringOwnership = new HashMap<>();
+ return constrainedAssign(partitionsPerTopic,
consumerToOwnedPartitions);
+ } else {
+ log.debug("Detected that all not consumers were subscribed to same
set of topics, falling back to the "
+ + "general case assignment algorithm");
+ partitionsTransferringOwnership = null;
Review comment:
I didn't bother to include this optimization for the general case. We
know that the assignment algorithm itself becomes a bottleneck at only 2,000
partitions, so there's no point optimizing something that only becomes a
bottleneck in the millions of partitions
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
