Re: Highlighter that works with phrase and span queries

2007-08-29 Thread Mike Klaas
I just meant whether it would live in a lucene release (somewhere under contrib/) or just in JIRA. Would including the functionality in Solr help get it into lucene? -Mike On 29-Aug-07, at 4:58 AM, Mark Miller wrote: It kind of is a contrib -- its really just a new Scorer class (with som

Re: Highlighter that works with phrase and span queries

2007-08-29 Thread Mark Miller
The patch you refer to should include the javadoc/source code. If that is not sufficient, drop me a line privately and I will email you all of the source code / javadoc. - Mark Michael Stoppelman wrote: Ah, much clearer now. It seems that the jar file is just the class files. Is the source/ja

Re: Highlighter that works with phrase and span queries

2007-08-29 Thread Mark Miller
It kind of is a contrib -- its really just a new Scorer class (with some axillary helper classes) for the old contrib Highlighter. Since the contrib Highlighter is pretty hardened at this point, I figured that was the best way to go. Or do you mean something different? - Mark Mike Klaas wrote

Re: Highlighter that works with phrase and span queries

2007-08-27 Thread Mike Klaas
Mark, I'm still interested in integrating this into Solr--this is a feature that has been requested a few times. It would be easier to do so if it were a contrib/... thanks for the great work, -Mike On 27-Aug-07, at 4:21 AM, Mark Miller wrote: I am a bit unclear about your question. The

Re: Highlighter that works with phrase and span queries

2007-08-27 Thread Michael Stoppelman
Ah, much clearer now. It seems that the jar file is just the class files. Is the source/javadoc code somewhere else? -M On 8/27/07, Mark Miller <[EMAIL PROTECTED]> wrote: > > I am a bit unclear about your question. The patch you mention extends > the original Highlighter to support phrase and spa

Re: Highlighter that works with phrase and span queries

2007-08-27 Thread Mark Miller
I am a bit unclear about your question. The patch you mention extends the original Highlighter to support phrase and span queries. It does not include any major performance increases over the original Highlighter (in fact, it takes a bit longer to Highlight a Span or Phrase query than it does t

Re: Highlighter that works with phrase and span queries

2007-08-27 Thread Michael Stoppelman
Is this jar going to be in the next release of lucene? Also, are these the same as the changes in the following patch: https://issues.apache.org/jira/secure/attachment/12362653/spanhighlighter10.patch -M On 6/27/07, Mark Miller <[EMAIL PROTECTED]> wrote: > > > > I have not looked at any highlight

Re: Highlighter that works with phrase and span queries

2007-06-27 Thread Mark Miller
I have not looked at any highlighting code yet. Is there already an extension of PhraseQuery that has getSpans() ? Currently I am using this code originally by M. Harwood: Term[] phraseQueryTerms = ((PhraseQuery) query).getTerms(); int i; SpanQuery[] clauses

Re: Highlighter that works with phrase and span queries

2007-06-27 Thread Paul Elschot
On Wednesday 27 June 2007 17:17, mark harwood wrote: > >>you would still have the major problem of which matches do you keep information for > > Yes, doing this efficiently is the main issue. Some vague thoughts I had: >... > 3) For each call to scorer.next() on the top level query, the Highligh

Re: Highlighter that works with phrase and span queries

2007-06-27 Thread mark harwood
h for just these few docs. Cheers Mark - Original Message From: Mark Miller <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, 27 June, 2007 12:59:21 PM Subject: Re: Highlighter that works with phrase and span queries markharw00d wrote: > > I was t

Re: Highlighter that works with phrase and span queries

2007-06-27 Thread Mark Miller
apache.org Sent: Wednesday, June 20, 2007 12:39:27 AM Subject: Highlighter that works with phrase and span queries I have been working on extending the Highlighter with a new Scorer that correctly scores phrase and span queries. The highlighter is working great for me, but could really use some

Re: Highlighter that works with phrase and span queries

2007-06-27 Thread Mark Miller
markharw00d wrote: I was thinking along the lines of wrapping some core classes such as IndexReader to somehow observe the query matching process and deduce from that what to highlight (avoiding the need for MemoryIndex) but I'm not sure that is viable. It would be nice to get some more ma

Re: Highlighter that works with phrase and span queries

2007-06-22 Thread Otis Gospodnetic
11:19:22 PM Subject: Re: Highlighter that works with phrase and span queries Results of my tests : The new SpanScorer is just about the same speed as the old Highlighter's QueryScorer if the Query contains no position sensitive elements. This is only the case, however, if the CachingTokenF

Re: Highlighter that works with phrase and span queries

2007-06-21 Thread markharw00d
Hi Mark, Good summary. I was running some timings earlier and my results echo your findings. >>I am currently trying to think of some possible hybrid approach to highlighting... I was thinking along the lines of wrapping some core classes such as IndexReader to somehow observe the query mat

Re: Highlighter that works with phrase and span queries

2007-06-21 Thread Mark Miller
From: Mark Miller <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Thursday, 21 June, 2007 2:11:52 AM Subject: Re: Highlighter that works with phrase and span queries I will work up some performance numbers over the next day or two to share with you. I have spent the last day or

Re: Highlighter that works with phrase and span queries

2007-06-21 Thread mark harwood
ng "helper" interfaces (Fragmenter/Scorer) remains to be seen. Cheers, Mark - Original Message From: Mark Miller <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Thursday, 21 June, 2007 2:11:52 AM Subject: Re: Highlighter that works with phrase and span queries I wil

Re: Highlighter that works with phrase and span queries

2007-06-20 Thread Mark Miller
I will work up some performance numbers over the next day or two to share with you. I have spent the last day or two with a profiler trying to find the biggest performance drains. Unfortunately, I will probably not be able to squeeze out much more performance than the current Highlighter. When

Re: Highlighter that works with phrase and span queries

2007-06-20 Thread Yonik Seeley
On 6/20/07, Chris Lu <[EMAIL PROTECTED]> wrote: Agree. But I think another reason why highlighting is slow could also be the need to retrieve the document's content, quite likely it's on the hard drive, which usually takes around 10ms for each small document, more for larger document. I'm not

Re: Highlighter that works with phrase and span queries

2007-06-20 Thread Chris Lu
er <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, June 20, 2007 12:39:27 AM Subject: Highlighter that works with phrase and span queries I have been working on extending the Highlighter with a new Scorer that correctly scores phrase and span queries. The highlighter is worki

Re: Highlighter that works with phrase and span queries

2007-06-20 Thread Otis Gospodnetic
er <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Wednesday, June 20, 2007 12:39:27 AM Subject: Highlighter that works with phrase and span queries I have been working on extending the Highlighter with a new Scorer that correctly scores phrase and span queries. The highlighter is

Re: Highlighter that works with phrase and span queries

2007-06-20 Thread Mike Klaas
On 19-Jun-07, at 3:39 PM, Mark Miller wrote: I have been working on extending the Highlighter with a new Scorer that correctly scores phrase and span queries. The highlighter is working great for me, but could really use some more banging on. If you have a need or an interest in a more accu

Highlighter that works with phrase and span queries

2007-06-19 Thread Mark Miller
I have been working on extending the Highlighter with a new Scorer that correctly scores phrase and span queries. The highlighter is working great for me, but could really use some more banging on. If you have a need or an interest in a more accurate Highlighter, please give it a whirl and let