[
https://issues.apache.org/jira/browse/KAFKA-13472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Egerton updated KAFKA-13472:
----------------------------------
Fix Version/s: 3.1.0
3.0.1
> Connect can lose track of last committed offsets for topic partitions after
> partial consumer revocation
> -------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-13472
> URL: https://issues.apache.org/jira/browse/KAFKA-13472
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Affects Versions: 3.1.0, 3.0.1
> Reporter: Chris Egerton
> Assignee: Chris Egerton
> Priority: Blocker
> Fix For: 3.1.0, 3.0.1
>
>
> The Connect framework tracks the last successfully-committed offsets for each
> topic partition that is currently assigned to the consumer of each sink task.
> If a sink task throws an exception from {{{}SinkTask::preCommit{}}}, the
> consumer is "rewound" by seeking to those last successfully-committed offsets
> for each topic partition, so that the same records can be redelivered to the
> task again.
> With the changes from KAFKA-12487, we failed to correctly update the logic
> for tracking these last-committed offsets which can cause topic partitions to
> be missing from them after partial revocation of topic partitions from the
> consumer. Specifically, we make the assumption that, whenever an offset
> commit succeeds, the offsets that were successfully committed constitute the
> entirely of the last-committed offsets for the task; when a partial consumer
> revocation takes place, we only commit offsets for some of the topic
> partitions assigned to the task's producer, and this assumption fails.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
