[
https://issues.apache.org/jira/browse/KAFKA-19451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17987010#comment-17987010
]
Jun Rao commented on KAFKA-19451:
---------------------------------
The test call the following before checking if the deleted files are removed.
{code:java}
TestUtils.waitForCondition(cacheEntry::isCleanStarted, "Failed to cleanup cache
entry after remove");{code}
However, in RemoteIndexCache, the index files are deleted after cleanStarted
is set to true.
{code:java}
void cleanup() throws IOException {
entryLock.writeLock().lock();
try {
// no-op if clean is done already
if (!cleanStarted && isMarkedForCleanup()) {
cleanStarted = true;
List<StorageAction<Void, Exception>> actions = List.of(() -> {
offsetIndex.deleteIfExists();
return null;
}, () -> {
timeIndex.deleteIfExists();
return null;
}, () -> {
txnIndex.deleteIfExists();
return null;
});
tryAll(actions);
}
{code}
> Fix flaky test: RemoteIndexCacheTest.testCacheEntryIsDeletedOnRemoval()
> -----------------------------------------------------------------------
>
> Key: KAFKA-19451
> URL: https://issues.apache.org/jira/browse/KAFKA-19451
> Project: Kafka
> Issue Type: Improvement
> Reporter: Jun Rao
> Priority: Major
>
> Saw the following flaky test in
> https://github.com/apache/kafka/actions/runs/15937660290/job/44960938007?pr=19961.
> {code:java}
> 2025-06-28T00:09:57.4324761Z Gradle Test Run :storage:test > Gradle Test
> Executor 47 > RemoteIndexCacheTest > testCacheEntryIsDeletedOnRemoval()
> STARTED 2025-06-28T00:09:57.4327688Z
> org.apache.kafka.storage.internals.log.RemoteIndexCacheTest.testCacheEntryIsDeletedOnRemoval()
> failed, log available in
> /home/runner/work/kafka/kafka/storage/build/reports/testOutput/org.apache.kafka.storage.internals.log.RemoteIndexCacheTest.testCacheEntryIsDeletedOnRemoval().test.stdout
> 2025-06-28T00:09:57.4329862Z 2025-06-28T00:09:57.4330480Z Gradle Test Run
> :storage:test > Gradle Test Executor 47 > RemoteIndexCacheTest >
> testCacheEntryIsDeletedOnRemoval() FAILED 2025-06-28T00:09:57.4332568Z
> java.io.UncheckedIOException: java.nio.file.NoSuchFileException:
> /tmp/kafka-RemoteIndexCacheTest6205787022310961862/remote-log-index-cache/2147584984_WMUkQgEpTJ6GksATEZn4iQ.txnindex.deleted
> 2025-06-28T00:09:57.4334299Z at
> java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:87)
> 2025-06-28T00:09:57.4336792Z at
> java.base/java.nio.file.FileTreeIterator.hasNext(FileTreeIterator.java:103)
> 2025-06-28T00:09:57.4337909Z at
> java.base/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1855)
> 2025-06-28T00:09:57.4339038Z at
> java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:129)
> 2025-06-28T00:09:57.4340181Z at
> java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:527)
> 2025-06-28T00:09:57.4341199Z at
> java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:513)
> 2025-06-28T00:09:57.4342209Z at
> java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
> 2025-06-28T00:09:57.4343223Z at
> java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
> 2025-06-28T00:09:57.4344159Z at
> java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> 2025-06-28T00:09:57.4345155Z at
> java.base/java.util.stream.ReferencePipeline.findAny(ReferencePipeline.java:652)
> 2025-06-28T00:09:57.4346710Z at
> org.apache.kafka.storage.internals.log.RemoteIndexCacheTest.getIndexFileFromRemoteCacheDir(RemoteIndexCacheTest.java:1290)
> 2025-06-28T00:09:57.4348431Z at
> org.apache.kafka.storage.internals.log.RemoteIndexCacheTest.testCacheEntryIsDeletedOnRemoval(RemoteIndexCacheTest.java:339)
> 2025-06-28T00:09:57.4349345Z 2025-06-28T00:09:57.4349471Z Caused
> by: 2025-06-28T00:09:57.4350789Z java.nio.file.NoSuchFileException:
> /tmp/kafka-RemoteIndexCacheTest6205787022310961862/remote-log-index-cache/2147584984_WMUkQgEpTJ6GksATEZn4iQ.txnindex.deleted
> 2025-06-28T00:09:57.4352393Z at
> java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:92)
> 2025-06-28T00:09:57.4353392Z at
> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
> 2025-06-28T00:09:57.4354360Z at
> java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
> 2025-06-28T00:09:57.4355793Z at
> java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
> 2025-06-28T00:09:57.4356539Z at
> java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:148)
> 2025-06-28T00:09:57.4357193Z at
> java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
> 2025-06-28T00:09:57.4357721Z at
> java.base/java.nio.file.Files.readAttributes(Files.java:1851)
> 2025-06-28T00:09:57.4358201Z at
> java.base/java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:220)
> 2025-06-28T00:09:57.4358697Z at
> java.base/java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:277)
> 2025-06-28T00:09:57.4359159Z at
> java.base/java.nio.file.FileTreeWalker.next(FileTreeWalker.java:374)
> 2025-06-28T00:09:57.4359670Z at
> java.base/java.nio.file.FileTreeIterator.fetchNextIfNeeded(FileTreeIterator.java:83)
> 2025-06-28T00:09:57.4360085Z ... 11 more
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
