[
https://issues.apache.org/jira/browse/KAFKA-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nandini Singhal reassigned KAFKA-19971:
---------------------------------------
Assignee: Nandini Singhal
> Tiered Storage Fetches Fail with IllegalStateException During Index Cache
> Eviction Window
> -----------------------------------------------------------------------------------------
>
> Key: KAFKA-19971
> URL: https://issues.apache.org/jira/browse/KAFKA-19971
> Project: Kafka
> Issue Type: Bug
> Components: Tiered-Storage
> Reporter: Nandini Singhal
> Assignee: Nandini Singhal
> Priority: Major
>
> Remote log fetches can fail with IllegalStateException: This entry is marked
> for cleanup when the index cache is under pressure and evicting entries. A
> race condition exists between entry eviction and fetch operations, causing
> requests to fail even though the data could be re-fetched from remote storage.
> When the remote index cache reaches its capacity and needs to evict entries,
> a multi-step eviction process creates a window where in-flight fetch requests
> can fail unexpectedly:
> 1. Cache eviction triggers removal of an index entry
> 2. Entry is marked for cleanup (markedForCleanup = true)
> 3. Index files are renamed to .deleted suffix
> 4. Cleanup is scheduled to run after a delay (default: 10 seconds)
> 5. After the delay, cleanup sets cleanStarted = true and deletes files
> 6. During this window, if a fetch request references this entry, it fails
> with:
> java.lang.IllegalStateException: This entry is marked for cleanup
>
> Expected behavior: Should treat as cache miss and re-fetch from remote
> storage (or handle gracefully).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
