Kirill Tkalenko created IGNITE-25180:
----------------------------------------
Summary: Fix race between drop index and partition garbage
collection
Key: IGNITE-25180
URL: https://issues.apache.org/jira/browse/IGNITE-25180
Project: Ignite
Issue Type: Bug
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
In the tests, an error was found in the partition garbage collection related to
indexes.
After analyzing the error and the code, I found out that we have a race between
dropping the index and garbage collecting, this needs to be fixed.
I think that it is worth doing the same as subsequent work with indexes, if
some index was not found, then it was deleted and we simply skip it.
This error is unlikely, and I managed to reproduce it for a table with 250+
indexes, and not the first time.
{noformat}
[09:55:36]W: [:ignite-table:integrationTest]
[2025-04-12T05:55:36,409][ERROR][%iet_n_3345%mv-gc-8][FailureManager] Critical
system error detected. Will be handled accordingly to configured handler
[hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED,
SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=CRITICAL_ERROR]
[09:55:36]W: [:ignite-table:integrationTest]
org.apache.ignite.internal.failure.FailureManager$StackTraceCapturingException:
java.lang.AssertionError: QualifiedName[schemaName='PUBLIC', objectName='TEST']
part 1
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:155)
~[ignite-failure-handler-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.failure.FailureManager.process(FailureManager.java:132)
~[ignite-failure-handler-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.gc.MvGc.lambda$scheduleGcForStorage$8(MvGc.java:234)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:863)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:841)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:614)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:653)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:482)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.lang.Thread.run(Thread.java:833) [?:?]
[09:55:36]W: [:ignite-table:integrationTest] Caused by:
java.util.concurrent.CompletionException: java.lang.AssertionError:
QualifiedName[schemaName='PUBLIC', objectName='TEST'] part 1
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
~[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
~[?:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:649)
~[?:?]
[09:55:36]W: [:ignite-table:integrationTest] ... 4 more
[09:55:36]W: [:ignite-table:integrationTest] Caused by:
java.lang.AssertionError: QualifiedName[schemaName='PUBLIC', objectName='TEST']
part 1
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.IndexWrapper$SortedIndexWrapper.getStorage(IndexWrapper.java:63)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.TableImpl.lambda$indexStorageAdapters$1(TableImpl.java:256)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.index.IndexUpdateHandler.indexStorageById(IndexUpdateHandler.java:253)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.index.IndexUpdateHandler.indexesSnapshot(IndexUpdateHandler.java:197)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.index.IndexUpdateHandler.tryRemoveFromIndexes(IndexUpdateHandler.java:104)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.gc.GcUpdateHandler.internalVacuum(GcUpdateHandler.java:148)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.gc.GcUpdateHandler.lambda$internalVacuumBatch$1(GcUpdateHandler.java:108)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.storage.pagememory.mv.PersistentPageMemoryMvPartitionStorage.lambda$runConsistently$0(PersistentPageMemoryMvPartitionStorage.java:187)
~[ignite-storage-page-memory-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.storage.pagememory.mv.AbstractPageMemoryMvPartitionStorage.busy(AbstractPageMemoryMvPartitionStorage.java:967)
~[ignite-storage-page-memory-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.storage.pagememory.mv.PersistentPageMemoryMvPartitionStorage.runConsistently(PersistentPageMemoryMvPartitionStorage.java:177)
~[ignite-storage-page-memory-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.storage.ThreadAssertingMvPartitionStorage.runConsistently(ThreadAssertingMvPartitionStorage.java:47)
~[ignite-storage-api-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.raft.snapshot.SnapshotAwarePartitionDataStorage.runConsistently(SnapshotAwarePartitionDataStorage.java:80)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.gc.GcUpdateHandler.internalVacuumBatch(GcUpdateHandler.java:102)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.gc.GcUpdateHandler.vacuumBatch(GcUpdateHandler.java:80)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
org.apache.ignite.internal.table.distributed.gc.MvGc.lambda$scheduleGcForStorage$7(MvGc.java:226)
~[ignite-table-9.0.127-SNAPSHOT.jar:?]
[09:55:36]W: [:ignite-table:integrationTest] at
java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)
~[?:?]
[09:55:36]W: [:ignite-table:integrationTest] ... 4 more
{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
