Re: Sorting posting lists before intersection

2008-10-13 Thread Renaud Delbru
Hi, Paul Elschot wrote: This could be done, but since not all scorers will be TermScorers it will be necessary to add a method to Scorer (or perhaps even to its DocIdSetIterator superclass): public abstract int estimatedDocFreq(); and implement this for all existing instances. TermScorer co

Re: Sorting posting lists before intersection

2008-10-13 Thread Renaud Delbru
Andrzej Bialecki wrote: Renaud Delbru wrote: Hi Andrzej, sorry for the late reply. I have looked at the code. As far as I understand, you sort the posting lists based on the first doc skip. The first posting list will be the one who have the first biggest document skip. Do the sparseness of

Re: Sorting posting lists before intersection

2008-10-13 Thread Paul Elschot
Op Monday 13 October 2008 17:00:06 schreef Andrzej Bialecki: > Renaud Delbru wrote: > > Hi Andrzej, > > > > sorry for the late reply. > > > > I have looked at the code. As far as I understand, you sort the > > posting lists based on the first doc skip. The first posting list > > will be the one who

Re: Sorting posting lists before intersection

2008-10-13 Thread Andrzej Bialecki
Renaud Delbru wrote: Hi Andrzej, sorry for the late reply. I have looked at the code. As far as I understand, you sort the posting lists based on the first doc skip. The first posting list will be the one who have the first biggest document skip. Do the sparseness of posting lists is a good p

Re: Sorting posting lists before intersection

2008-10-13 Thread Renaud Delbru
Hi Andrzej, sorry for the late reply. I have looked at the code. As far as I understand, you sort the posting lists based on the first doc skip. The first posting list will be the one who have the first biggest document skip. Do the sparseness of posting lists is a good predictor for sampling

Re: Sorting posting lists before intersection

2008-09-17 Thread Jason Rutherglen
It would be a good feature in Lucene to be able to sort, or perhaps store the postings in term frequency sorted order. Thoughts? On Wed, Sep 17, 2008 at 9:33 AM, Andrzej Bialecki <[EMAIL PROTECTED]> wrote: > Renaud Delbru wrote: >> >> Hi all, >> >> I am wondering if Lucene implements the query op

Re: Sorting posting lists before intersection

2008-09-17 Thread Andrzej Bialecki
Renaud Delbru wrote: Hi all, I am wondering if Lucene implements the query optimisation that consists of ordering the posting lists based on the term frequency before intersection ? If yes, could somebody point me to the java class / method that implements such strategy ? Lucene trunk: Conj

Sorting posting lists before intersection

2008-09-17 Thread Renaud Delbru
Hi all, I am wondering if Lucene implements the query optimisation that consists of ordering the posting lists based on the term frequency before intersection ? If yes, could somebody point me to the java class / method that implements such strategy ? Thanks in advance, Regards. -- Renaud De