Jack, To demonstrate this, I had set up a simple test, measuring the impact of search quality (Mean Average Precision) against % of an index filtered. The test was setup by taking the TREC 2004 collection, breaking that up into several smaller chunks of increasing size, then measured the MAP of these chunks using topics and quality judgements from Robust track 2004. Finally I repeated the measurements against the complete collection, this time I took several measurements of the index filtered against the corresponding chunk using QueryWrapperFilter. The results of difference between the two measurements showed quite significant change in MAP values, I would have expected there to be no change if relevance scores were calculated based on filtered document count, instead of system wide term stats. See results here : http://goo.gl/BI4fv
Of course, this bug/feature leads to some interesting exploits, particularly in the context of securing the search space by filtering search results. Nigel On 1 July 2013 13:09, Jack Krupansky <j...@basetechnology.com> wrote: > The very definition of a "filter" in Lucene is that it doesn't influence > relevance/scoring in any way, so your question is a contradiction in terms. > > If you are finding that the use of a filter is affecting the scores of > documents, then that is clearly a bug. > > -- Jack Krupansky > > -----Original Message----- From: Nigel V Thomas > Sent: Monday, July 01, 2013 7:38 AM > To: java-user@lucene.apache.org > Subject: Relevance ranking calculation based on filtered document count > > > Hi, > > I would like to know if it is possible to calculate the relevance ranks of > documents based on filtered document count? > The current filter implementations as far as I know, seems to be applied > after the query is processed and ranked against the full set of documents. > Since system wide IDF values are used to rank documents, the resulting > ordering is different from a set whose range is restricted only to the > filtered set of documents. > > Many thanks, > > Nigel > > ------------------------------**------------------------------**--------- > To unsubscribe, e-mail: > java-user-unsubscribe@lucene.**apache.org<java-user-unsubscr...@lucene.apache.org> > For additional commands, e-mail: > java-user-help@lucene.apache.**org<java-user-h...@lucene.apache.org> >
