Heres what I mean:
http://lucene.apache.org/java/docs/queryparsersyntax.html#Fields
title:"The Right Way" AND text:go
Although, I am not searching for the title "the right way" , I am looking
for the score by specifying a unique field (itemID).
when I do System.out.println(query);
I get:
+contents:Harvard +contents:Business + contents: Review
Can I just add:
+contents:Harvard +contents:Business + contents: Review +itemID=id ??
That query would just return one document.
On 7/25/07, Askar Zaidi <[EMAIL PROTECTED]> wrote:
>
> Instead of refactoring the code, would there be a way to just modify the
> query in each search routine ?
>
> Such as, "search contents:<text> and item:<itemID>"; This means it would
> just collect the score of that one document whose itemID field = itemID
> passed from while( rs.next()).
>
> I just need to collect the score of the <itemID> already in the index.
>
> Would there be a way to modify the query ? Add a clause ?
>
> thanks,
> Askar
>
>
> On 7/25/07, Grant Ingersoll <[EMAIL PROTECTED]> wrote:
> >
> > So, you really want a single Lucene score (based on the scores of
> > your 4 fields) for every itemID, correct? And this score consists of
> > scoring the title, tag, summary and body against some keywords correct?
> >
> > Here's what I would do:
> >
> > while (rs.next())
> > {
> > doc = getDocument(itemId); // Get your document, including
> > contents from your database, no need even to put them in Lucene,
> > although you could
> > add the doc to a MemoryIndex (see contrib/memory)
> > Run your 4 searches against that memory index to get your
> > score. Even better, combine your query into a single query that
> > searches all 4 fields at once, then Lucene will combine the score for
> > you
> > }
> >
> > MemoryIndex info can be found at http://lucene.zones.apache.org:8080/
> > hudson/job/Lucene-Nightly/javadoc/org/apache/lucene/index/memory/
> > package-summary.html
> >
> > -Grant
> >
> > On Jul 25, 2007, at 11:45 AM, Askar Zaidi wrote:
> >
> > > Hi Grant,
> > >
> > > Thanks for the response. Heres what I am trying to accomplish:
> > >
> > > 1. Iterate over itemID (unique) in the database using one SQL query.
> > > 2. For every itemID found, run 4 searches on Lucene Index.
> > > 3. doTagSearch(itemID....) ; collect score
> > > 4. doTitleSearch(itemID...) ; collect score
> > > 5. doSummarySearch(itemID...) ; collect score
> > > 6. doBodySearch(itemID....) ; collect score
> > >
> > > These scores are then added and I get a total score for each unique
> > > item in
> > > the database.
> > >
> > > Lucene Index has: <itemID><tags><title><summary><contents>
> > >
> > > So if I am running a body search, I have 92 hits from over 300
> > > documents for
> > > a query. I already know my hit with the <itemID> .
> > >
> > > For instance, from step (1) if itemID 16 is passed to all the 4
> > > searches, I
> > > just need to get the score of the document which has itemID field =
> > > 16. I
> > > don't have to iterate over all the hits.
> > >
> > > I suppose I have to change my query to look for <contents> where
> > > itemID=16.
> > > Can you guide me as to how to do it ?
> > >
> > > thanks a ton,
> > >
> > > Askar
> > >
> > > On 7/25/07, Grant Ingersoll <[EMAIL PROTECTED] > wrote:
> > >>
> > >> Hi Askar,
> > >>
> > >> I suggest we take a step back, and ask the question, what are you
> > >> trying to accomplish? That is, what is your application trying to
> > >> do? Forget the code, etc. just explain what you want the end result
> > >> to be and we can work from there. Based on what you have described,
> > >> I am not sure you need access to the hits. It seems like you just
> > >> need to make better queries.
> > >>
> > >> Is your itemID a unique identifier? If yes, then you shouldn't need
> > >> to loop over hits at all, as you should only ever have one result IF
> > >> your query contains a required term. Also, if this is the case, why
> > >> do you need to do a search at all? Haven't you already identified
> > >> the items of interest when you did your select query in the
> > >> database? Or is it that you want to score the item based on some
> > >> terms as well. If that is the case, there are other ways of doing
> > >> this and we can discuss them.
> > >>
> > >> -Grant
> > >>
> > >> On Jul 25, 2007, at 10:10 AM, Askar Zaidi wrote:
> > >>
> > >>> Hey Guys,
> > >>>
> > >>> I need to know how I can use the HitCollector class ? I am using
> > >>> Hits and
> > >>> looping over all the possible document hits (turns out its 92 times
> > >>> I am
> > >>> looping; for 300 searches, its 300*92 !!). Can I avoid this using
> > >>> HitCollector ? I can't seem to understand how its used.
> > >>>
> > >>> thanks a lot,
> > >>>
> > >>> Askar
> > >>>
> > >>> On 7/25/07, Dmitry <[EMAIL PROTECTED]> wrote:
> > >>>>
> > >>>> Askar,
> > >>>> why do you need to add +id:<idWeCareAbout>?
> > >>>> thanks,
> > >>>> dt,
> > >>>> www.ejinz.com
> > >>>> search engine news forms
> > >>>> ----- Original Message -----
> > >>>> From: "Askar Zaidi" <[EMAIL PROTECTED] >
> > >>>> To: <java-user@lucene.apache.org>; <[EMAIL PROTECTED]>
> > >>>> Sent: Wednesday, July 25, 2007 12:39 AM
> > >>>> Subject: Re: Fine Tuning Lucene implementation
> > >>>>
> > >>>>
> > >>>>> Hey Hira ,
> > >>>>>
> > >>>>> Thanks so much for the reply. Much appreciate it.
> > >>>>>
> > >>>>> Quote:
> > >>>>>
> > >>>>> Would it be possible to just include a query clause?
> > >>>>> - i.e., instead of just contents:<userQuery>, also add
> > >>>>> +id:<idWeCareAbout>
> > >>>>>
> > >>>>> How can I do that ?
> > >>>>>
> > >>>>> I see my query as :
> > >>>>>
> > >>>>> +contents:harvard +contents:business +contents:review
> > >>>>>
> > >>>>> where the search phrase was: harvard business review
> > >>>>>
> > >>>>> Now how can I add +id:<idWeCareAbout> ??
> > >>>>>
> > >>>>> This would give me that one exact document I am looking for , for
> > >>>>> that
> > >>>> id.
> > >>>>> I
> > >>>>> don't have to iterate through hits.
> > >>>>>
> > >>>>> thanks,
> > >>>>>
> > >>>>> Askar
> > >>>>>
> > >>>>>
> > >>>>>
> > >>>>> On 7/24/07, N. Hira < [EMAIL PROTECTED]> wrote:
> > >>>>>>
> > >>>>>> I'm no expert on this (so please accept the comments in that
> > >>>>>> context)
> > >>>>>> but 2 things seem weird to me:
> > >>>>>>
> > >>>>>> 1. Iterating over each hit is an expensive proposition. I've
> > >>>>>> often
> > >>>>>> seen people recommending a HitCollector.
> > >>>>>>
> > >>>>>> 2. It seems that doBodySearch() is essentially saying, do this
> > >>>>>> search
> > >>>>>> and return the score pertinent to this ID (using an exhaustive
> > >>>>>> loop).
> > >>>>>> Would it be possible to just include a query clause?
> > >>>>>> - i.e., instead of just contents:<userQuery>, also add
> > >>>>>> +id:<idWeCareAbout>
> > >>>>>>
> > >>>>>> In general though, I think your algorithm seems inefficient (if I
> > >>>>>> understand it correctly):-- if I want to search for one term
> > >>>>>> among 3 in
> > >>>>>> a "collection" of 300 documents (as defined by some external
> > >>>> attribute),
> > >>>>>> I will wind up executing 300 x 3 searches, and for each search
> > >>>>>> that is
> > >>>>>> executed, I will iterate over every Hit, even if I've already
> > >>>>>> found the
> > >>>>>> one that I "care about".
> > >>>>>>
> > >>>>>> What would break if you:
> > >>>>>> 1. Included "creator" in the Lucene index (or, filtered out the
> > >>>>>> Hits
> > >>>>>> using a BitSet or something like it)
> > >>>>>> 2. Executed 1 search
> > >>>>>> 3. Collected the results of the first N Hits (where N is some
> > >>>>>> reasonable limit, like 100 or 500)
> > >>>>>>
> > >>>>>> -h
> > >>>>>>
> > >>>>>>
> > >>>>>> On Tue, 2007-07-24 at 20:14 -0400, Askar Zaidi wrote:
> > >>>>>>
> > >>>>>>> Sure.
> > >>>>>>>
> > >>>>>>> public float doBodySearch(Searcher searcher,String query, int
> > >>>>>>> id){
> > >>>>>>>
> > >>>>>>> try{
> > >>>>>>> score = search(searcher,
> > >>>>>>> query,id);
> > >>>>>>> }
> > >>>>>>> catch(IOException io){}
> > >>>>>>> catch(ParseException pe){}
> > >>>>>>>
> > >>>>>>> return score;
> > >>>>>>>
> > >>>>>>> }
> > >>>>>>>
> > >>>>>>> private float search(Searcher searcher, String queryString,
> > >>>>>>> int id)
> > >>>>>>> throws ParseException, IOException {
> > >>>>>>>
> > >>>>>>> // Build a Query object
> > >>>>>>>
> > >>>>>>> QueryParser queryParser = new QueryParser("contents",
> > >>>>>>> new
> > >>>>>>> KeywordAnalyzer());
> > >>>>>>>
> > >>>>>>> queryParser.setDefaultOperator
> > >>>>>>> ( QueryParser.Operator.AND);
> > >>>>>>>
> > >>>>>>> Query query = queryParser.parse(queryString);
> > >>>>>>>
> > >>>>>>> // Search for the query
> > >>>>>>>
> > >>>>>>> Hits hits = searcher.search(query);
> > >>>>>>> Document doc = null;
> > >>>>>>>
> > >>>>>>> // Examine the Hits object to see if there were any
> > >>>>>>> matches
> > >>>>>>> int hitCount = hits.length();
> > >>>>>>>
> > >>>>>>> for(int i=0;i<hitCount;i++){
> > >>>>>>> doc = hits.doc(i);
> > >>>>>>> String str = doc.get("item");
> > >>>>>>> int tmp = Integer.parseInt (str);
> > >>>>>>> if(tmp==id)
> > >>>>>>> score = hits.score(i);
> > >>>>>>> }
> > >>>>>>>
> > >>>>>>> return score;
> > >>>>>>> }
> > >>>>>>>
> > >>>>>>> I really need to optimize doBodySearch(...) as this takes the
> > >>>>>>> most
> > >>>>>>> time.
> > >>>>>>>
> > >>>>>>> thanks guys,
> > >>>>>>> Askar
> > >>>>>>>
> > >>>>>>>
> > >>>>>>> On 7/24/07, N. Hira <[EMAIL PROTECTED]> wrote:
> > >>>>>>>
> > >>>>>>> Could you show us the relevant source from
> > >>>>>>> doBodySearch()?
> > >>>>>>>
> > >>>>>>> -h
> > >>>>>>>
> > >>>>>>> On Tue, 2007-07-24 at 19:58 -0400, Askar Zaidi wrote:
> > >>>>>>>> I ran some tests and it seems that the slowness is from
> > >>>>>>> Lucene calls when I
> > >>>>>>>> do "doBodySearch", if I remove that call, Lucene gives me
> > >>>>>>> results in 5
> > >>>>>>>> seconds. otherwise it takes about 50 seconds.
> > >>>>>>>>
> > >>>>>>>> But I need to do Body search and that field contains lots
> > >>>> of
> > >>>>>>> text. The field
> > >>>>>>>> is <contents>. How can I optimize that ?
> > >>>>>>>>
> > >>>>>>>> thanks,
> > >>>>>>>> Askar
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>>
> > >>>>>
> > >>>>
> > >>>>
> > >>>> -------------------------------------------------------------------
> > >>>> --
> > >>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
> > >>>> For additional commands, e-mail: [EMAIL PROTECTED]
> > >>>>
> > >>>>
> > >>
> > >> --------------------------
> > >> Grant Ingersoll
> > >> Center for Natural Language Processing
> > >> http://www.cnlp.org/tech/lucene.asp
> > >>
> > >> Read the Lucene Java FAQ at http://wiki.apache.org/lucene-java/
> > >> LuceneFAQ
> > >>
> > >>
> > >>
> > >> ---------------------------------------------------------------------
> >
> > >> To unsubscribe, e-mail: [EMAIL PROTECTED]
> > >> For additional commands, e-mail: [EMAIL PROTECTED]
> > >>
> > >>
> >
> > ------------------------------------------------------
> > Grant Ingersoll
> > http://www.grantingersoll.com/
> > http://lucene.grantingersoll.com
> > http://www.paperoftheweek.com/
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
