Dale Lane created FLINK-40619:
---------------------------------
Summary:
DynamicKafkaSourceReaderTest.testHandleSourceEventRetainsRemovedClusterOffsetsUntilExpired
flaky - removed-cluster retention expiry cannot be tested deterministically
Key: FLINK-40619
URL: https://issues.apache.org/jira/browse/FLINK-40619
Project: Flink
Issue Type: Bug
Components: Connectors / Kafka
Reporter: Dale Lane
{{DynamicKafkaSourceReaderTest.testHandleSourceEventRetainsRemovedClusterOffsetsUntilExpired}}
fails intermittently under CPU contention with
{{java.util.NoSuchElementException: No value present
at java.util.Optional.orElseThrow(Optional.java:377)
at
DynamicKafkaSourceReaderTest.testHandleSourceEventRetainsRemovedClusterOffsetsUntilExpired(DynamicKafkaSourceReaderTest.java:294)}}
(It passes consistently on an unloaded developer machine, but fails
intermittently in pull request workflows.)
The behaviour under test is time-based. When a cluster is removed,
{{DynamicKafkaSourceReader}} gives the affected splits a deadline of "now plus
the configured retention", and {{snapshotState()}} prunes splits whose deadline
has passed before returning them.
The test configures a retention window of one second and then asserts against
the retained split across several {{snapshotState()}} calls. The whole sequence
must therefore complete within one second of wall-clock time. A garbage
collection or scheduling pause longer than that expires the split before the
assertions read it, and the lookup finds nothing.
The reader reads the current time directly from the system clock at each of the
three points where retention is stamped or evaluated. There is no way for a
test to control the passage of time, so an expiry test has to pick a real-time
window and race it. The short window is fragile when running in the Github CI
workers under load (and increasing it to a long window would not assert that
expiry behaviour is really correct).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
