Re: prorated early termination

2019-02-03 Thread Michael McCandless
On Sun, Feb 3, 2019 at 10:41 AM Michael Sokolov wrote: > > In single-threaded mode we can check against minCompetitiveScore and > terminate collection for each segment appropriately, > > > Does Lucene do this today by default? That should be a nice > optimization, > and it'd be safe/correct. >

Re: prorated early termination

2019-02-03 Thread Michael Sokolov
> > In single-threaded mode we can check against minCompetitiveScore and terminate collection for each segment appropriately, > Does Lucene do this today by default? That should be a nice optimization, and it'd be safe/correct. Yes, it does that today (in TopFieldCollector -- see https://github

Re: prorated early termination

2019-02-03 Thread Michael McCandless
I think this is because our per-hit cost is sometimes very high -- we have "post filters" that are sometimes very restrictive. We are working to get those post-filters out into an inverted index to make them more efficient, but net/net reducing how many hits we must collect for each segment can he

Re: prorated early termination

2019-02-03 Thread Michael McCandless
One question about this: > In single-threaded mode we can check against minCompetitiveScore and terminate collection for each segment appropriately, Does Lucene do this today by default? That should be a nice optimization, and it'd be safe/correct. Mike McCandless http://blog.mikemccandless.co