[
https://issues.apache.org/jira/browse/FLINK-40081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40081:
-----------------------------------
Labels: pull-request-available (was: )
> Enable the spilling backend for checkpointing during recovery
> -------------------------------------------------------------
>
> Key: FLINK-40081
> URL: https://issues.apache.org/jira/browse/FLINK-40081
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Checkpointing
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Major
> Labels: pull-request-available
>
> Swap the memory backend for the disk backend behind
> execution.checkpointing.during-recovery.enabled: recovered channel state is
> fetched (and rescale-filtered) into spill files, the drainer pushes it into
> the recovery-state channels while serving as the recovery-checkpoint trigger,
> and checkpoints taken during recovery snapshot the not-yet-drained slice.
> Scope:
> * Handler factory returns the spilling handlers when the flag is on; the v1
> in-memory filtering path is deleted.
> * StreamTask flag-on recovery becomes fetch → requestPartitions →
> build/install drainer → drain → NO_OP; the transitional in-memory trigger and
> conversion-time push are deleted.
> * The unbounded heap fallback in RecoveredInputChannel#requestBufferBlocking
> is removed — spilling supersedes it, eliminating the OOM risk of the memory
> backend.
> * RecordFilterContext hardened for minimal and batch environments (spill tmp
> directories enforced only when the flag is on).
> * IT cases: unaligned checkpoint during recovery, rescale-filter
> large-record OOM regression, non-source recovery race smoke test.
> The randomized ITCase fleet (which runs the flag in ~50% of runs) must stay
> green across the backend swap — the acceptance bar for spilling being a
> drop-in replacement of the memory backend.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
