Hello, thanks for your detailed response. I didn't know there was a method called setBoost for adjusting the relevance of a certain document. Now I simply calculate the boosting factor for the document, based on its newness, the sales rank and some other values.
Thank you very much. Best regards, Daniel -----Original Message----- From: Grant Ingersoll [mailto:[EMAIL PROTECTED] Sent: Saturday, April 26, 2008 12:42 AM To: java-user@lucene.apache.org Subject: Re: boosting relevance of certain documents It really depends. Hand tuning scoring algs for a specific query is very prone to local maxima problems. In other words, you fix one query and break 50 others. Sometimes, a good old "configurable" hard code is the way to go. If you want a specific doc to be #1, make it number one. You will pull your hair out otherwise. In Solr, this is handled via the Query Elevation Component, but isn't all that difficult to implement. Likewise, if you have a priori knowledge that a particular document is more important, then give it a relatively large boost during indexing, being aware that Lucene does not offer much granularity in terms of boosts. In other words, boost it something like 5 or 10 times, instead of 1.1 vs. 1.2. On the other hand, if you are truly seeing broad problems, then you need to build up a set of queries and judgments (ala TREC) or the contrib/benchmark Quality packages. You might also look at Lucene's Similarity class. Lucene's length normalization is often less than optimal for certain types of documents (see the IBM Haifa's assessment for the "Million Query" track of TREC on the Lucene Wiki). Cheers, Grant On Apr 25, 2008, at 3:50 PM, Daniel Freudenberger wrote: > Thanks for your response. I already knew that the relevance is based > on the > term frequency but in some cases it's just not what the user expects. > As I already mentioned, "fifa 2003 fifa 03" vs. "fifa 08" is such a > case - > searching for "fifa" would return the "fifa 2003 fifa 03" document > first but > the "fifa 08" document is more important (from the user's point of > view). > > Any suggestions? > > Best regards, > Daniel > -----Original Message----- > From: Jonathan Ariel [mailto:[EMAIL PROTECTED] > Sent: Friday, April 25, 2008 8:11 PM > To: java-user@lucene.apache.org > Subject: Re: boosting relevance of certain documents > > Ok. So I'm not an expert of the scoring algorithm, but based on > tf*idf you > can tell that the returned document is more relevant because it has > more > term frequency. > > Using the explain you can see the following: > > Doc 1 > 0.643841 = (MATCH) fieldWeight(searchable:fifa in 0), product of: > 1.0 = tf(termFreq(searchable:fifa)=1) > 1.287682 = idf(docFreq=2) > 0.5 = fieldNorm(field=searchable, doc=0) > > Doc2 > 0.68289655 = (MATCH) fieldWeight(searchable:fifa in 1), product of: > 1.4142135 = tf(termFreq(searchable:fifa)=2) > 1.287682 = idf(docFreq=2) > 0.375 = fieldNorm(field=searchable, doc=1) > > On Fri, Apr 25, 2008 at 2:30 PM, Daniel Freudenberger < > [EMAIL PROTECTED]> wrote: > >> I'm using the StandardAnalyzer - hope this answers your question (I'm >> quite >> new to the lucene thing) >> >> -----Original Message----- >> From: Jonathan Ariel [mailto:[EMAIL PROTECTED] >> Sent: Friday, April 25, 2008 6:59 PM >> To: java-user@lucene.apache.org >> Subject: Re: boosting relevance of certain documents >> >> How are you analyzing the searchable field? >> >> On Fri, Apr 25, 2008 at 12:49 PM, Daniel Freudenberger < >> [EMAIL PROTECTED]> wrote: >> >>> Hello, >>> >>> >>> >>> I'm using lucene within a new project and I'm not sure about how to >> solve >>> the following problem: My index consists of the two attributes >>> "id" and >>> "searchable". "id" is the id of a product and "searchable" is a >>> combination >>> of the product name and its category name. >>> >>> >>> >>> example: >>> >>> id searchable >>> >>> 1 fifa 08 - playstation 3 >>> >>> 2 fifa 2003 fifa 03 - playstation 3 >>> >>> 3 playstation 60gb hdd - playstation 3 >>> >>> 4 playstation i like you - playstation 3 >>> >>> >>> >>> When searching for "fifa", lucene returns the product with id 2 at >> first, >>> whereas id 1 ("fifa 08") would be the much more relevant result >>> (from >> the >>> user side of view). the same problem arises when searching for >>> "playstation" >>> - the customer expects products having "playstation" in their >>> names at >>> first, ideally the console itself. in reality however, he gets all >>> possible >>> products which are in the "playstation" category as well. >>> >>> >>> >>> my idea was to introduce another attribute relevance, which may >>> increase >>> the >>> relevance of an entry. the actual relevance shouldn't be suppressed >>> completely though, but should only be taken into account with >>> products >>> that >>> are similarly relevant for a specific search term. >>> >>> >>> >>> Does anybody have an idea on how to solve this problem? >>> >>> >>> >>> Thank you in advance, >>> >>> Daniel >>> >>> >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [EMAIL PROTECTED] >> For additional commands, e-mail: [EMAIL PROTECTED] >> >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > -------------------------- Grant Ingersoll Lucene Helpful Hints: http://wiki.apache.org/lucene-java/BasicsOfPerformance http://wiki.apache.org/lucene-java/LuceneFAQ --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
