I've been able to reproduce a crash we are seeing in our product with
newer Lucene versions.
I'm attaching a small Java code that reproduces this. It might look
weird, it's the result of removing every custom thing we are applying to
the query while still seeing the bug.
This is the crash I see with this code (with assertions disabled it
crashes in a different place):
Exception in thread "main" java.lang.AssertionError
at
org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)
at
org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)
at
org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)
at
org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)
at
org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)
at
org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)
at
org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)
at
org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)
at
org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)
at
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
at
org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)
at
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)
at
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)
at
org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)
at
org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)
at
org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)
at
org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)
at
org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)
at
org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:661)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:574)
at
org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:421)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:432)
at com.wolfram.textsearch.LuceneCrash.main(LuceneCrash.java:48)
Interestingly, the bug does not happen if the index is created on a
ByteBuffersDirectory.
I hope this is useful!
Thanks!
package com.wolfram.textsearch;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.TextField;
import org.apache.lucene.index.*;
import org.apache.lucene.queries.function.FunctionScoreQuery;
import org.apache.lucene.search.*;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;
import java.io.IOException;
import java.nio.file.Paths;
public class LuceneCrash
{
public static void main(String[] args)
{
Directory dir;
try
{
// not reproduced with new ByteBuffersDirectory!
dir = FSDirectory.open(Paths.get("/tmp/xxx"));
IndexWriter indexWriter = new IndexWriter(dir, new IndexWriterConfig());
Document doc = new Document();
doc.add(new TextField("ExampleText", "function", Field.Store.NO));
indexWriter.addDocument(doc);
indexWriter.commit();
indexWriter.close();
} catch (IOException e)
{
e.printStackTrace();
return;
}
try (DirectoryReader reader = DirectoryReader.open(dir))
{
Query q;
BooleanQuery.Builder bq = new BooleanQuery.Builder();
bq.add(new TermQuery(new Term("ExampleText", "function")), BooleanClause.Occur.SHOULD);
bq.add(new TermQuery(new Term("ExampleText", "plot")), BooleanClause.Occur.SHOULD);
q = bq.build();
q = FunctionScoreQuery.boostByQuery(q, new PhraseQuery(1, "ExampleText", "function", "plot"), 2);
q = FunctionScoreQuery.boostByValue(q, DoubleValuesSource.SCORES);
System.out.println(new IndexSearcher(reader).search(q, 10).totalHits);
} catch (IOException e)
{
e.printStackTrace();
}
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org