MartijnVisser opened a new pull request, #28686:
URL: https://github.com/apache/flink/pull/28686
## What is the purpose of the change
FLINK-32695 (umbrella) migrates test code off the deprecated
`SourceFunction` API to Source V2 (FLIP-27). Roughly 15 checkpointing ITCases
depend on a legacy `FailingSource` pattern: a source with a checkpointed
position that fails exactly once after a completed checkpoint and resumes from
the restored position. No Source V2 test utility offered these semantics.
This PR adds `FailingCheckpointedSource` to `flink-test-utils-connector` and
migrates `EventTimeAllWindowCheckpointingITCase` as the first consumer. The
remaining consumers follow in separate PRs under the umbrella.
## Brief change log
- Add `FailingCheckpointedSource` + `SequenceSplit` (a position-carrying
split) to `flink-test-utils-connector`:
- A snapshot past the arming threshold arms the failure; emission then
pauses at the failure position; completion of the armed checkpoint triggers the
throw; an aborted armed checkpoint re-arms on a later snapshot.
- The reader holds emission just past the arming threshold until a
snapshot arms, so recovery deterministically replays the arming-to-failure
window regardless of emission speed or state backend (the legacy source relied
on 1ms-per-element pacing for this; at full V2 speed the armed checkpoint would
otherwise capture the failure position itself, losing detection power).
- Recovery attempts are detected via the restored non-zero split position
(`SourceReaderContext` exposes no attempt number).
- The enumerator assigns one split per subtask only on first registration
(a surviving enumerator on partial failover must not double-assign).
- `withSimulatedStateLossOnRecovery()` exists purely so consuming tests
can validate that they still detect exactly-once violations. Rescaling is
unsupported (documented).
- Migrate `EventTimeAllWindowCheckpointingITCase` off `FailingSource`.
`KeyedEventTimeGenerator` (in the not-yet-migrated
`EventTimeWindowCheckpointingITCase`) temporarily implements both generator
interfaces so it can be shared; that file is touched only for this reason.
## Verifying this change
- 14 unit tests in `FailingCheckpointedSourceTest` cover the failure
protocol, arming threshold, abort re-arm, restore-without-refailing, enumerator
assignment/regression paths, and the serializer.
- `EventTimeAllWindowCheckpointingITCase` passes 4/4.
- Red check: with `withSimulatedStateLossOnRecovery()` temporarily wired in,
`testTumblingTimeWindow` fails on duplicated window sums (e.g. `[Window start:
1300 end: 1400] expected: 134950`), proving the migrated test still detects
exactly-once violations. Reverted afterwards.
## Notes for reviewers
- **New classes are annotated `@Experimental`** (author decision: test
utilities should not carry cross-minor evolution guarantees yet; the module's
pre-existing `@PublicEvolving` classes are untouched). Per AGENTS.md, new
stability annotations warrant committer input, so feedback here is welcome,
this also sets the pattern for the sibling Source-V2 test utilities that follow
in the umbrella.
- `flink-core`'s test-jar is added as a test-scope dependency of
`flink-test-utils-connector` (for `MockSplitEnumeratorContext`).
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): **no** (test-scope
only: flink-core test-jar)
- The public API, i.e., is any changed class annotated with
`@Public(Evolving)`: **no** (new classes are `@Experimental`, test-scoped)
- The serializers: **no**
- The runtime per-record code paths (performance sensitive): **no**
- Anything that affects deployment or recovery: JobManager (and its
components), Checkpointing, Kubernetes/Yarn, ZooKeeper: **no** (test-only)
- The S3 file system connector: **no**
## Documentation
- Does this pull request introduce a new feature? **no** (test
infrastructure)
- If yes, how is the feature documented? **not applicable**
---
<sub>🤖 This change was drafted with AI assistance (Claude Code) and reviewed
by the author before submission. Commits carry a `Generated-by` trailer per the
project's AI-contribution convention.</sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
