How did you arrive at that MAP? What analyzers, etc.? So much of
search depends on your choices during indexing and querying, etc.
There is some work by the IBM Haifa people up on the Wiki, so that
would be one place to check. Another question is what is your end
goal is in doing a TREC evaluation? Are you competing in TREC, or are
you just trying to get a feel for how Lucene performs? If it is the
latter, TREC is just one data point in the relevance sea and you can
definitely risk overtuning for TREC. You will definitely need a means
for evaluating your data set.
And yes, most evals. I have seen pay attention to things like: pseudo
relevance feedback and other expansion techniques, query construction,
as well as adjusting Lucene's length normalization (see the Similarity
class.) You might also have a look at the contrib/benchmark Quality
package. Finally, one of these days, I am hoping we will get some
TREC collections for use by committers on ASF hardware. This process
has been started and hopefully will be available soon.
Cheers,
Grant
On May 4, 2008, at 2:13 PM, DanaWhite wrote:
Hello all,
I have been doing some evaluation of Lucene on a TReC collection and
get a
rather disappointing mean average precision (MAP) of 11%. Other
sources
seem to report a MAP of about 20%.
So I am here to ask all of you who have done some Lucene evaluation
- what
did you find Lucene's MAP to be? Did you do anything to special
(e.g. query
expansion) to get it higher?
Thanks
Dana
--
View this message in context:
http://www.nabble.com/Lucene%27s-Mean-Average-Precision-tp17049412p17049412.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--------------------------
Grant Ingersoll
Lucene Helpful Hints:
http://wiki.apache.org/lucene-java/BasicsOfPerformance
http://wiki.apache.org/lucene-java/LuceneFAQ
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
