Re: Slower search after 8.5.x to >=8.6

2021-04-08 Thread Никита Михайлов
> It's usually more a bottleneck when indexing with IndexWriter # > updateDocument, which needs to perform one ID lookup for every indexed > document. I guess that the queries that you are running match so few hits that very little time is spent reading postings, is that correct? > But then that

Slower search after 8.5.x to >=8.6

2021-04-08 Thread Никита Михайлов
Hi. I noticed that after the upgrade from Lucene8.5.x to Lucene >=8.6, search became slower(example TopScoreDocCollector became 20-30% slower, from ElasticSearch - 50%). While testing, I realized that it happened after LUCENE-9257(commit e7a61ea). Bug or feature? Can add settings for isOffHeep? T

Re: Slower search after 8.5.x to >=8.6

2021-04-08 Thread Adrien Grand
FSDirectory#open is just a utility method that tries to pick the best Directory implementation based on the platform, it's most likely MMapDirectory for you, which is the directory implementation we use on all 64-bit platforms. So it's intriguing that you are seeing a slowdown with MMapDirectory bu

Re: Slower search after 8.5.x to >=8.6

2021-04-08 Thread Никита Михайлов
Thanks for the answer NIOFSDirectory is like an example. Degradation is also on MMapDirectory and SimpleFSDirectory We are using elasticseach and it has: simplefs (SimpleFsDirectory), niofs (NIOFSDirectory), mmapfs (MMapDirectory) and hybridfs (NIOFSDirectory + MMapDirectory). And for us, while ni

Re: Slower search after 8.5.x to >=8.6

2021-04-07 Thread Adrien Grand
Hello, Why are you forcing NIOFSDirectory instead of using Lucene's defaults via FSDirectory#open? I wonder if this might contribute to the slowdown you are seeing given that access to the terms index tends to be a bit random. It's very unlikely we'll add back a toggle for this as there is no poi

Slower search after 8.5.x to >=8.6

2021-04-07 Thread Никита Михайлов
Hi. I noticed that after the upgrade from Lucene8.5.x to Lucene >=8.6, search became slower(example TopScoreDocCollector became 20-30% slower, from ElasticSearch - 50%). While testing, I realized that it happened after LUCENE-9257(commit e7a61ea). Bug or feature? Can add settings for isOffHeep? T