lianetm commented on code in PR #18089:
URL: https://github.com/apache/kafka/pull/18089#discussion_r1879280407
##########
clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractMembershipManager.java:
##########
@@ -1175,21 +1175,8 @@ private CompletableFuture<Void> assignPartitions(
// Invoke user call back.
CompletableFuture<Void> result =
signalPartitionsAssigned(addedPartitions);
- result.whenComplete((__, exception) -> {
- if (exception == null) {
- // Enable newly added partitions to start fetching and
updating positions for them.
-
subscriptions.enablePartitionsAwaitingCallback(addedPartitions);
- } else {
- // Keeping newly added partitions as non-fetchable after the
callback failure.
- // They will be retried on the next reconciliation loop, until
it succeeds or the
- // broker removes them from the assignment.
- if (!addedPartitions.isEmpty()) {
- log.warn("Leaving newly assigned partitions {} marked as
non-fetchable and not " +
- "requiring initializing positions after
onPartitionsAssigned callback failed.",
- addedPartitions, exception);
- }
- }
- });
+ // Enable newly added partitions to start fetching and updating
positions for them.
+ result.whenComplete((__, exception) ->
subscriptions.enablePartitionsAwaitingCallback(addedPartitions));
Review Comment:
I see, and agree we you there @chia7712, the 2nd round of callbacks will
include empty partitions (on both consumers), but then isn't the bug still on
the recovery (meaning, re-enabling the partitions that were awaiting
callbacks)?
On the new consumer we "enable" only the added partitions (empty on the 2nd
round of callbacks), but we still have "assignedPartitions" that will remain
awaitingCallbacks so never fetchable. This line:
https://github.com/apache/kafka/blob/3cf8745243026b83d97fbc80f2b1fe03f1455701/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AbstractMembershipManager.java#L1181
Shouldn't we make sure we `enablePartitionsAwaitingCallback` for all
assignedPartitions instead of only the added ones? and that's what will allow
to resume fetching
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
