These failures relate to the way Arrays.binarySearch works when there are repeated values, in which case the result is undefined (it can be any of the indexes with the value), but in SlowCompositeReaderWrapper we are relying on finding the lowest-indexed of the repeats. I'll work on a fix
On Sat, Sep 28, 2024 at 3:32 PM Policeman Jenkins Server <[email protected]> wrote: > > Build: https://jenkins.thetaphi.de/job/Lucene-main-Linux/50364/ > Java: 64bit/hotspot/jdk-24-ea+16 -XX:-UseCompressedOops -XX:+UseParallelGC > > 2 tests failed. > FAILED: > org.apache.lucene.backward_codecs.lucene90.TestLucene90HnswVectorsFormat.testSparseVectors > > Error Message: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > > Stack Trace: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > at > app/[email protected]/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:916) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:929) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1550) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1838) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1478) > at > app/[email protected]/org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:224) > at > app/[email protected]/org.apache.lucene.tests.index.BaseKnnVectorsFormatTestCase.testSparseVectors(BaseKnnVectorsFormatTestCase.java:873) > at > java.base@24-ea/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base@24-ea/java.lang.reflect.Method.invoke(Method.java:573) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) > at > app/[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/[email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at > app/[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at > app/[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at > app/[email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) > at java.base@24-ea/java.lang.Thread.run(Thread.java:1576) > Caused by: java.lang.NullPointerException: Cannot invoke > "org.apache.lucene.index.FloatVectorValues.vectorValue(int)" because > "this.subs[this.lastSubIndex].sub" is null > at > [email protected]/org.apache.lucene.index.SlowCompositeCodecReaderWrapper$SlowCompositeKnnVectorsReaderWrapper$MergedFloatVectorValues.vectorValue(SlowCompositeCodecReaderWrapper.java:905) > at > [email protected]/org.apache.lucene.index.SortingCodecReader$SortingFloatVectorValues.vectorValue(SortingCodecReader.java:324) > at > [email protected]/org.apache.lucene.codecs.KnnVectorsWriter$MergedVectorValues$MergedFloat32VectorValues.vectorValue(KnnVectorsWriter.java:364) > at > org.apache.lucene.backward_codecs.lucene90.Lucene90HnswVectorsWriter.writeVectorData(Lucene90HnswVectorsWriter.java:194) > at > org.apache.lucene.backward_codecs.lucene90.Lucene90HnswVectorsWriter.writeField(Lucene90HnswVectorsWriter.java:122) > at > [email protected]/org.apache.lucene.codecs.BufferingKnnVectorsWriter.mergeOneField(BufferingKnnVectorsWriter.java:203) > at > [email protected]/org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.mergeOneField(PerFieldKnnVectorsFormat.java:121) > at > [email protected]/org.apache.lucene.codecs.KnnVectorsWriter.merge(KnnVectorsWriter.java:105) > at > [email protected]/org.apache.lucene.index.SegmentMerger.mergeVectorValues(SegmentMerger.java:271) > at > [email protected]/org.apache.lucene.index.SegmentMerger.mergeWithLogging(SegmentMerger.java:314) > at > [email protected]/org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158) > at > [email protected]/org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5283) > at > [email protected]/org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4749) > at > [email protected]/org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:6567) > at > [email protected]/org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:668) > at > [email protected]/org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:729) > > > FAILED: > org.apache.lucene.backward_codecs.lucene92.TestLucene92HnswVectorsFormat.testSparseVectors > > Error Message: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > > Stack Trace: > org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed > at > app/[email protected]/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:916) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:929) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1550) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1838) > at > app/[email protected]/org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1478) > at > app/[email protected]/org.apache.lucene.tests.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:224) > at > app/[email protected]/org.apache.lucene.tests.index.BaseKnnVectorsFormatTestCase.testSparseVectors(BaseKnnVectorsFormatTestCase.java:873) > at > java.base@24-ea/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) > at java.base@24-ea/java.lang.reflect.Method.invoke(Method.java:573) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48) > at > app/[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/[email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891) > at > app/[email protected]/com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902) > at > app/[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) > at > app/[email protected]/org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60) > at > app/[email protected]/org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47) > at > app/[email protected]/org.junit.rules.RunRules.evaluate(RunRules.java:20) > at > app/[email protected]/com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390) > at > app/[email protected]/com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850) > at java.base@24-ea/java.lang.Thread.run(Thread.java:1576) > Caused by: java.lang.NullPointerException: Cannot invoke > "org.apache.lucene.index.FloatVectorValues.vectorValue(int)" because > "this.subs[this.lastSubIndex].sub" is null > at > [email protected]/org.apache.lucene.index.SlowCompositeCodecReaderWrapper$SlowCompositeKnnVectorsReaderWrapper$MergedFloatVectorValues.vectorValue(SlowCompositeCodecReaderWrapper.java:905) > at > [email protected]/org.apache.lucene.index.SortingCodecReader$SortingFloatVectorValues.vectorValue(SortingCodecReader.java:324) > at > [email protected]/org.apache.lucene.codecs.KnnVectorsWriter$MergedVectorValues$MergedFloat32VectorValues.vectorValue(KnnVectorsWriter.java:364) > at > org.apache.lucene.backward_codecs.lucene92.Lucene92HnswVectorsWriter.writeVectorData(Lucene92HnswVectorsWriter.java:197) > at > org.apache.lucene.backward_codecs.lucene92.Lucene92HnswVectorsWriter.writeField(Lucene92HnswVectorsWriter.java:129) > at > [email protected]/org.apache.lucene.codecs.BufferingKnnVectorsWriter.mergeOneField(BufferingKnnVectorsWriter.java:203) > at > [email protected]/org.apache.lucene.codecs.perfield.PerFieldKnnVectorsFormat$FieldsWriter.mergeOneField(PerFieldKnnVectorsFormat.java:121) > at > [email protected]/org.apache.lucene.codecs.KnnVectorsWriter.merge(KnnVectorsWriter.java:105) > at > [email protected]/org.apache.lucene.index.SegmentMerger.mergeVectorValues(SegmentMerger.java:271) > at > [email protected]/org.apache.lucene.index.SegmentMerger.mergeWithLogging(SegmentMerger.java:314) > at > [email protected]/org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:158) > at > [email protected]/org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:5283) > at > [email protected]/org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4749) > at > [email protected]/org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:6567) > at > [email protected]/org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:668) > at > [email protected]/org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:729) > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
