[
https://issues.apache.org/jira/browse/FLINK-40586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40586:
-----------------------------------
Labels: pull-request-available (was: )
> DynamicKafkaSourceEnumerator re-signals NoMoreSplits to already-finished
> readers on every reader registration, causing "OperatorEvent lost" task
> failover
> ---------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-40586
> URL: https://issues.apache.org/jira/browse/FLINK-40586
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: kafka-3.4.0, kafka-4.0.1, kafka-5.0.0, kafka-5.1.0
> Reporter: Martijn Visser
> Priority: Major
> Labels: pull-request-available
>
> DynamicKafkaSourceITTest$IntegrationTests.testIdleReader fails in the weekly
> CI with
> org.apache.flink.util.FlinkException: An OperatorEvent from an
> OperatorCoordinator to a task was
> lost. Triggering task failover to ensure consistency. Event:
> '[NoMoreSplitEvent]', targetTask:
> Source: Tested Source (3/5) - execution #0
> Caused by:
> org.apache.flink.runtime.operators.coordination.TaskNotRunningException: Task
> is not
> running, but in state FINISHED
> e.g. run 33283936010 job 99183711637 (2026-08-30, main, JDK 11) and run
> 29667371405 job 88140088972 (2026-07-19, v5.0, Flink 2.1.2, JDK 21); also
> seen 2026-07-05 (v4.0) and 2026-08-02 (v5.0/2.2.1).
> DynamicKafkaSourceEnumerator.handleNoMoreSplits() calls
> enumContext.signalNoMoreSplits for every registered reader on every
> invocation, and it is invoked from addReader, addSplitsBack and
> tryCompletePendingReaderRegistration without remembering which readers were
> already signalled. testIdleReader uses parallelism = number of splits + 1, so
> the idle reader finishes as soon as it receives the first signal; the next
> reader registration re-sends NoMoreSplitsEvent to the FINISHED task, the
> event is not loss-tolerant, and the coordinator fails the job.
> This is distinct from FLINK-40362, which fixes the reader-side hang where an
> idle reader swallows the signal when it arrives before the
> MetadataUpdateEvent. That fix makes idle readers finish earlier and, if
> anything, widens this window.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
