Rui Fan created FLINK-40080:
-------------------------------
Summary: Drain spilled channel state into input channels and
replay spilled slices into checkpoints
Key: FLINK-40080
URL: https://issues.apache.org/jira/browse/FLINK-40080
Project: Flink
Issue Type: Sub-task
Components: Runtime / Checkpointing
Reporter: Rui Fan
Assignee: Rui Fan
The disk-side counterpart of the recovery-checkpoint coordination. Runtime
behavior is unchanged — no production path constructs a drainer, and the
handler factory does not select the spilling handlers.
Scope:
* RecoveryCheckpointTrigger#snapshotAndInsertBarriers widened to return a
FetchedChannelStateReader: checkpoint start atomically snapshots the
not-yet-drained spill slice and inserts recovery barriers.
* ChannelStateWriter#addInputDataFromSpill: the channel-state writer replays a
spilled slice into the checkpoint
(ChannelStateWriteRequest.replayInputDataFromSpill,
ChannelStateCheckpointWriter.writeInputFromSpill, streaming serializer
overload); reader ownership transfers to the writer, closed on cancel/failure.
* ChannelState checkpoint-start dispatch gains step 3: hand the snapshot
reader to the writer; failure path closes the snapshot and routes
CheckpointException to abort.
* FetchedChannelStateDrainer: incremental drain of spill segments into
RecoverableInputChannels (buffer request + disk read outside the lock, delivery
+ offset advance inside), implementing the trigger with atomic
snapshot-and-insert; appends the end-of-recovery sentinels when the drain
finishes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
