Re: Multiple merge-runs from same set of segments

Yeah, like you said, it looks quite tough to load entire FST in-mem. Plus we have to address concurrent access I guess I have to first see how the current patch goes before any optimisations are done Thanks for the help! — Ravi On Thu, 27 May 2021 at 10:44 PM, Patrick Zhai wrote: > Sorry for

Re: Multiple merge-runs from same set of segments

Sorry for the delayed response, as for caching termDict data across threads, I do not aware of any existing lucene mechanism could do that (and it might be tricky since it is across threads), but maybe worth trying to see whether we can get some extra speed based on that! Patrick Ravikumar Govind

Re: Multiple merge-runs from same set of segments

Thanks Patrick for the help! May I know what lucene version you're using? > We are using an older version of lucene as of now (4.7.x) and I believe the FilterCodecReader of current version is akin to FilterAtomicReader & should do the job for us! If it is not available, I'm not sure whether the

Re: Multiple merge-runs from same set of segments

Hi Ravi, 1. May I know what lucene version you're using? As far as I know the SortingMergePolicy has been deprecated and replaced by IndexWriterConfig.setIndexSort in newer lucene version. So if the "setIndexSort" is available I would suggest using that to achieve the sorted index (as you might ha

Re: Multiple merge-runs from same set of segments

Thanks Michael! This was just what I was looking for!!. Just a couple of questions. - When we call addIndexes(IndexReader...), does the merge happen via MergePolicy? We use a SortingMergePolicy and would like to maintain the sort-order in newly created segments too - Concurrency is a

Re: Multiple merge-runs from same set of segments

Are you trying to rewrite your already created index into a different segment geometry? Maybe have a look at the new IndexRearranger tool ? It is already doing something like what you enumerated below, including mocking LiveDocs to get the right