Hi All
Thank you for all your suggestions. Some of the recommendations hadn't
yet been implemented, as our code base was using older versions of
Lucene with reduced capabilities. Thus, far, all the recommendations
for fast search have been implemented (e.g. using pagination with
searchAfter, DirectoryReader.openIfChanged, avoiding wrapping lucene
scoreDoc results, option to disable sorting, etc.).
While, in some environments, search performance has improved
significantly, in other larger ones we are unfortunately, still seeing 1
minute - 5 minute search times. For instance, in one site, the total
index size is 500GB with 190 million documents indexed. They are running
a machine with 24 core and 4 SSD drives to house the indexes. New emails
are being added to the indexes at a rate of 10 message/sec.
One area possible area for improvement: Searching is being conducted
across several indexes. To accomplish this, on each search, a
MultiReader is constructed, that consists of several subreaders created
by the DirectoryReader.openIfChangedMethod. Only one of the indexes is
updated frequently, the others are never updated. For each search, a
new IndexSearcher is created passed the MultiReader in the constructor.
From what I've read, MultiReader and IndexSearcher are relatively
lightweight and should not impact search performance. Is this correct?
Is there a faster way to handle searching across multiple indexes? What
is the performance impact of searching across multiple indexes?
Am I correct that using SearchManager can't be used with a MultiReader
and NRT? I would appreciate all suggestions on how to optimize our
search performance further. Search time has become a usability issue.
Much appreciate
Jamie
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
