I think the merge itself isn't actually getting skipped, it's a
thread-safety issue in the test.
The test collects InfoStream messages into a ArrayList<String>, but with
ConcurrentMergeScheduler, the "merged new segment" message gets written by
the merge background thread. One thread writes to a non-thread-safe list,
the other test thread reads, oops, visibility problem in JMM. To address,
simply change:
List<String> infoStream = new ArrayList<>();
to:
List<String> infoStream = Collections.synchronizedList(new ArrayList<>());
I can reproduce this by putting the test in a for-loop running 10k times,
it shows roughly 0.3% chance of failure.
To verify the theory, I quickly create a JCStress test (a dummy one though,
but it illustrates the point). It shows the write from one thread is
completely or partially invisible to the reader thread:
@JCStressTest
@State
@Outcome(id = "1, 42", expect = Expect.ACCEPTABLE, desc = "Fully visible
write.")
@Outcome(id = "0, -1", expect = Expect.ACCEPTABLE, desc = "Write not
visible at all.")
@Outcome( expect = Expect.ACCEPTABLE_INTERESTING, desc =
"Partial / stale visibility.")
public class ArrayListVisibilityTest {
final List<Integer> list = new ArrayList<>();
@Actor
public void writer() {
list.add(42);
}
@Actor
public void reader(II_Result r) {
int size = list.size();
r.r1 = size;
try {
r.r2 = size > 0 ? list.get(0) : -1;
} catch (Exception e) {
// ArrayIndexOutOfBoundsException is possible under races
r.r2 = -2;
}
}
}
```
RESULT SAMPLES FREQ EXPECT DESCRIPTION
0, -1 1,838,722,183 67.78% Acceptable Write not visible at all.
1, -2 106,178 <0.01% Interesting Partial / stale
visibility.
1, 42 873,770,813 32.21% Acceptable Fully visible write.
```
In practice, a lock (or a memory barrier) would guarantee visibility, in
essence, the merge thread's CPU flushes writes to store buffer, and the
test thread's CPU is able to snoop the invalid to get the freshly added
message (at least this should be how things work on x86).
Thanks,
neoremind
Michael Sokolov <[email protected]> 于2026年4月6日周一 09:52写道：
> I just added this TestInfoStream.testMergeSmallIndex test so I thought
> I'd try to reproduce, but it didn't repro for me. The failure
> indicates that no segment merge was logged. I'm not sure how it could
> get skipped though.
>
> On Sun, Apr 5, 2026 at 1:57 PM Policeman Jenkins Server via builds
> <[email protected]> wrote:
> >
> > Build: https://jenkins.thetaphi.de/job/Lucene-MMAPv2-Linux/5230/
> > Java: 64bit/hotspot/jdk-27-ea+14 -XX:-UseCompressedOops -XX:+UseZGC
> >
> > 3 tests failed.
> > FAILED: org.apache.lucene.index.TestInfoStream.testMergeSmallIndex
> >
> > Error Message:
> > java.lang.AssertionError: expected:<1> but was:<0>
> >
> > Stack Trace:
> > java.lang.AssertionError: expected:<1> but was:<0>
> > at
> __randomizedtesting.SeedInfo.seed([F77FD74A101B3B0B:45303098296E3A19]:0)
> > at org.junit.Assert.fail(Assert.java:89)
> > at org.junit.Assert.failNotEquals(Assert.java:835)
> > at org.junit.Assert.assertEquals(Assert.java:647)
> > at org.junit.Assert.assertEquals(Assert.java:633)
> > at
> org.apache.lucene.index.TestInfoStream.testMergeSmallIndex(TestInfoStream.java:151)
> > at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:565)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1763)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> > at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:52)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > at java.base/java.lang.Thread.run(Thread.java:1527)
> >
> >
> > FAILED:
> org.apache.lucene.tests.store.TestMockDirectoryWrapper.testIsLoaded
> >
> > Error Message:
> > java.lang.AssertionError
> >
> > Stack Trace:
> > java.lang.AssertionError
> > at
> __randomizedtesting.SeedInfo.seed([F77FD74A101B3B0B:D95E8D3E0CAB8C7C]:0)
> > at org.junit.Assert.fail(Assert.java:87)
> > at org.junit.Assert.assertTrue(Assert.java:42)
> > at org.junit.Assert.assertTrue(Assert.java:53)
> > at
> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1663)
> > at
> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1623)
> > at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:565)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1763)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> > at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:52)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > at java.base/java.lang.Thread.run(Thread.java:1527)
> >
> >
> > FAILED:
> org.apache.lucene.tests.store.TestSerializedIOCountingDirectory.testIsLoaded
> >
> > Error Message:
> > java.lang.AssertionError
> >
> > Stack Trace:
> > java.lang.AssertionError
> > at
> __randomizedtesting.SeedInfo.seed([F77FD74A101B3B0B:D95E8D3E0CAB8C7C]:0)
> > at org.junit.Assert.fail(Assert.java:87)
> > at org.junit.Assert.assertTrue(Assert.java:42)
> > at org.junit.Assert.assertTrue(Assert.java:53)
> > at
> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1663)
> > at
> org.apache.lucene.tests.store.BaseDirectoryTestCase.testIsLoaded(BaseDirectoryTestCase.java:1623)
> > at
> java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
> > at java.base/java.lang.reflect.Method.invoke(Method.java:565)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1763)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
> > at
> org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
> > at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:52)
> > at
> org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
> > at
> org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
> > at
> org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
> > at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> > at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
> > at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
> > at java.base/java.lang.Thread.run(Thread.java:1527)
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [email protected]
> > For additional commands, e-mail: [email protected]
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [email protected]
> For additional commands, e-mail: [email protected]
>
>
--
Best regards,
Xu
