Lianet Magrans created KAFKA-20106:
--------------------------------------
Summary: Improve behaviour of consumer assignment API under async
reconciliations
Key: KAFKA-20106
URL: https://issues.apache.org/jira/browse/KAFKA-20106
Project: Kafka
Issue Type: Task
Components: clients, consumer
Reporter: Lianet Magrans
Assignee: Lianet Magrans
With the new rebalance protocol and the AsyncConsumer implementation, the
client reconciles assignments in the background as it received them from the
broker.
This leads to new interactions of the assignment API with others like seek() or
position(). Taking seek as an example, given that the assignment can change in
the background in between calls to assignment() and seek(), applications can
hit IllegalStateException "No current assignment for partition t0" on seek() if
the partition got unassigned. This is part of the contract of the seek() call,
but note that under the classic protocol and consumer, it requires a call to
poll to make anything change in the assignment (so seek could only fail in this
way if there is a call to poll in-between, but not on sequence of assignment()
+ seek())
One option is to change how to update the subscription state when a
reconciliation completes, so that even if the reconciliation completes in the
background it's not made effective until there is call to poll. Main
consideration here would be if there is any impact, e.g atm partitions become
fetchable as soon as the reconciliation completes, so wonder if this would
change/affect the partitions that get added to a fetch request in the
background (even though we only generate fetch requests on the foreground on
poll)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
