Re: Relevancy tests

2014-06-12 Thread Doug Turnbull
Relevancy judgement lists ARE very context sensitive. For example, in a medical search application you'll have very different relevancy requirements between a point-of-care applications vs an application being used to perform general "sit at your desk" research ***even if the content being served i

Re: Relevancy tests

2014-06-12 Thread Ahmet Arslan
Hi, Relevance Judgments are labor intensive and expensive. Some Information Retrieval forums ( TREC, CLEF, etc) provide these golden sets. But they are not public. http://rosenfeldmedia.com/books/search-analytics/ talks about how to create a "golden set" for your top n queries. Also there ar

Relevancy tests

2014-06-12 Thread Ivan Brusic
Perhaps more of an NLP question, but are there any tests regarding relevance for Lucene? Given an example corpus of documents, what are the golden sets for specific queries? The Wikidump dump is used as a benchmarking tool for both indexing and querying in Lucene, but there are no metrics in terms