[
https://issues.apache.org/jira/browse/FLINK-40685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-40685:
-----------------------------------
Labels: pull-request-available (was: )
> ForSt skips local cache cleanup for checkpoint-owned files
> ----------------------------------------------------------
>
> Key: FLINK-40685
> URL: https://issues.apache.org/jira/browse/FLINK-40685
> Project: Flink
> Issue Type: Bug
> Components: Runtime / State Backends
> Reporter: Aaron He
> Priority: Major
> Labels: pull-request-available
>
> h2. Problem
> ForSt can keep a local cached copy of an SST file whose source is owned by a
> checkpoint.
> When the backend no longer references that file, it must preserve the
> checkpoint-owned source. However, the current cleanup also skips releasing
> the local cached copy.
> This can retain unnecessary files in the local cache. Unbounded disk growth
> and production impact have not been established.
> h2. Why this happens
> MappingEntry.referenceCountReachedZero() returns immediately when the file is
> marked NOT_OWNED. The local cache cleanup appears after that return, so it is
> never reached for these files.
> h2. Expected behavior
> Release the local cache entry when the final mapping reference is released,
> while preserving the checkpoint-owned source. Active readers must remain
> usable; physical deletion should follow the cache's existing
> reference-counting behavior.
> h2. Proposed fix
> Move eligible local cache cleanup before the ownership check. Keep the
> ownership check for deleting the source file, and preserve existing cleanup
> for backend-owned files.
> h2. Planned work
> I will work on the fix and add regression tests covering the ownership
> handoff and active readers.
> h2. Reproduction to add
> # Create an SST file and populate its local cache entry.
> # Transfer ownership using a FileStateHandle.
> # Release the final mapping reference.
> # Check that the cache entry is invalidated, while the checkpoint file
> remains readable.
> Also check that an active reader can finish safely and that physical
> cache-file deletion waits for its reference to be released.
> h2. Evidence
> The cleanup order is still present on Apache master at 07b4ff26f545,
> inspected on 2026-09-16. This finding is based on source inspection; no tests
> or runtime reproduction have been performed for this draft.
> [MappingEntry
> cleanup|https://github.com/apache/flink/blob/07b4ff26f545e0d9373949fd82573074ae442291/flink-state-backends/flink-statebackend-forst/src/main/java/org/apache/flink/state/forst/fs/filemapping/MappingEntry.java#L141-L157]
> h2. Proposed release note
> ForSt reclaims local cached copies when mappings to checkpoint-owned files
> are released.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
