I've been able to reproduce a crash we are seeing in our product with newer Lucene versions.

I'm attaching a small Java code that reproduces this. It might look weird, it's the result of removing every custom thing we are applying to the query while still seeing the bug.

This is the crash I see with this code (with assertions disabled it crashes in a different place):

Exception in thread "main" java.lang.AssertionError
    at org.apache.lucene.codecs.lucene84.Lucene84PostingsReader$EverythingEnum.nextPosition(Lucene84PostingsReader.java:940)     at org.apache.lucene.search.PhrasePositions.nextPosition(PhrasePositions.java:57)     at org.apache.lucene.search.PhrasePositions.firstPosition(PhrasePositions.java:46)     at org.apache.lucene.search.SloppyPhraseMatcher.initSimple(SloppyPhraseMatcher.java:368)     at org.apache.lucene.search.SloppyPhraseMatcher.initPhrasePositions(SloppyPhraseMatcher.java:356)     at org.apache.lucene.search.SloppyPhraseMatcher.reset(SloppyPhraseMatcher.java:153)     at org.apache.lucene.search.PhraseScorer$1.matches(PhraseScorer.java:49)     at org.apache.lucene.search.DoubleValuesSource$WeightDoubleValuesSource$1.advanceExact(DoubleValuesSource.java:631)     at org.apache.lucene.queries.function.FunctionScoreQuery$QueryBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:343)     at org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)     at org.apache.lucene.search.DoubleValues$1.advanceExact(DoubleValues.java:53)     at org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.advanceExact(FunctionScoreQuery.java:270)     at org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:228)     at org.apache.lucene.search.DoubleValuesSource$2.doubleValue(DoubleValuesSource.java:344)     at org.apache.lucene.search.DoubleValues$1.doubleValue(DoubleValues.java:48)     at org.apache.lucene.queries.function.FunctionScoreQuery$MultiplicativeBoostValuesSource$1.doubleValue(FunctionScoreQuery.java:265)     at org.apache.lucene.queries.function.FunctionScoreQuery$FunctionScoreWeight$1.score(FunctionScoreQuery.java:229)     at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:76)     at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:276)     at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:232)
    at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39)
    at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:661)     at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:445)     at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:574)     at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:421)     at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:432)
    at com.wolfram.textsearch.LuceneCrash.main(LuceneCrash.java:48)

Interestingly, the bug does not happen if the index is created on a ByteBuffersDirectory.

I hope this is useful!

Thanks!

package com.wolfram.textsearch;

import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.TextField;
import org.apache.lucene.index.*;
import org.apache.lucene.queries.function.FunctionScoreQuery;
import org.apache.lucene.search.*;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.FSDirectory;

import java.io.IOException;
import java.nio.file.Paths;

public class LuceneCrash
{
	public static void main(String[] args)
	{
		Directory dir;
		try
		{
			// not reproduced with new ByteBuffersDirectory!
			dir = FSDirectory.open(Paths.get("/tmp/xxx"));
			IndexWriter indexWriter = new IndexWriter(dir, new IndexWriterConfig());
			Document doc = new Document();
			doc.add(new TextField("ExampleText", "function", Field.Store.NO));
			indexWriter.addDocument(doc);
			indexWriter.commit();
			indexWriter.close();
		} catch (IOException e)
		{
			e.printStackTrace();
			return;
		}

		try (DirectoryReader reader = DirectoryReader.open(dir))
		{
			Query q;

			BooleanQuery.Builder bq = new BooleanQuery.Builder();
			bq.add(new TermQuery(new Term("ExampleText", "function")), BooleanClause.Occur.SHOULD);
			bq.add(new TermQuery(new Term("ExampleText", "plot")), BooleanClause.Occur.SHOULD);
			q = bq.build();

			q = FunctionScoreQuery.boostByQuery(q, new PhraseQuery(1, "ExampleText", "function", "plot"), 2);
			q = FunctionScoreQuery.boostByValue(q, DoubleValuesSource.SCORES);

			System.out.println(new IndexSearcher(reader).search(q, 10).totalHits);
		} catch (IOException e)
		{
			e.printStackTrace();
		}
	}
}

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Reply via email to