Re: 2.9 per segment searching/caching

2009-10-22 Thread John Wang
HI Michael: I understand exactly what you mean. I have done some experiments with the multiQ approach by carrying over the bottom to next segment. (which would need to extend the ScoreDocComparator api to support the same type of "convert", the difference here is that it is optional, sup

Re: 2.9 per segment searching/caching

2009-10-22 Thread Mark Miller
Yes - in many cases, the other wins outweigh the queue transition cost - in some cases it does not. But we are talking degradation as you add more segments, not pure speed. Degradation is worse now in the sort case. John Wang wrote: > With many other coding that happened in 2.9, e.g. the PQ api e

Re: 2.9 per segment searching/caching

2009-10-22 Thread John Wang
With many other coding that happened in 2.9, e.g. the PQ api etc., sorting is actually faster than 2.4. -John On Thu, Oct 22, 2009 at 5:07 AM, Mark Miller wrote: > Bill Au wrote: > > Since Lucene 2.9 has per segment searching/caching, does query > performance > > degrade less than before (2.9) a

Re: 2.9 per segment searching/caching

2009-10-22 Thread Mark Miller
Bill Au wrote: > Since Lucene 2.9 has per segment searching/caching, does query performance > degrade less than before (2.9) as more segments are added to the index? > Bill > > I think non sorting cases are actually faster now over multiple segments - though you will still see performance degrad

Re: 2.9 per segment searching/caching

2009-10-22 Thread Simon Willnauer
Bill, per segments search does not replace index optimisation neither it prevents the performance degrade if your number of segments is increasing. Depending on how your index changes it can give you a performance improvement when reopening the index and it will certainly prevent one or another GC

2.9 per segment searching/caching

2009-10-21 Thread Bill Au
Since Lucene 2.9 has per segment searching/caching, does query performance degrade less than before (2.9) as more segments are added to the index? Bill