Re: MultiPhraseQuery or PhraseQuery to take the synonyms into account?

2018-09-22 Thread Michael McCandless
PhraseQuery can indeed be used to represent a multi-token synonym. In fact, I mis-spoke before: MultiPhraseQuery can also represent a multi-token synonym when the multiple tokens are all the same except in one spot. Mike McCandless http://blog.mikemccandless.com On Thu, Sep 20, 2018 at 2:32

Re: MultiPhraseQuery or PhraseQuery to take the synonyms into account?

2018-09-20 Thread baris.kazar
i should have asked this way as Mike made clear for MultiPhraseQuery: is PhraseQuery ok to account for synonyms? Best > On Sep 20, 2018, at 2:02 PM, baris.ka...@oracle.com wrote: > > Hi,- > > should i use MultiPhraseQuery or PhraseQuery to take synonyms into account? &g

MultiPhraseQuery or PhraseQuery to take the synonyms into account?

2018-09-20 Thread baris . kazar
Hi,-  should i use MultiPhraseQuery or PhraseQuery to take synonyms into account? Best regards baris - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h

Re: MultiPhraseQuery

2018-09-18 Thread baris . kazar
/18/18 5:28 PM, Michael McCandless wrote: Yes, +1 for a patch to improve the docs! MultiPhraseQuery only works for single term synonyms, and is usually produced by query parsers when the incoming query text had single term synonyms matching, I think? The query parser will use other (span?) queries

Re: MultiPhraseQuery

2018-09-18 Thread Michael McCandless
Yes, +1 for a patch to improve the docs! MultiPhraseQuery only works for single term synonyms, and is usually produced by query parsers when the incoming query text had single term synonyms matching, I think? The query parser will use other (span?) queries for multi token synonyms. I think the

Re: MultiPhraseQuery

2018-09-18 Thread baris . kazar
FuzzyQuery seems also not suitable for me. PrefixQuery can be one token only, right? Best On 9/18/18 5:23 PM, baris.ka...@oracle.com wrote: Erick,-  i think the reason why MultiPhraseQuery was created was synonyms as far as i understood. am i right? i want to have a BooleanQuery or

Re: MultiPhraseQuery

2018-09-18 Thread baris . kazar
Erick,-  i think the reason why MultiPhraseQuery was created was synonyms as far as i understood. am i right? i want to have a BooleanQuery or MultiPhraseQuery (i cant decide between these two) with an index which considers synonyms already. One disadvantage of MultiPhraseQuery is that it

Re: MultiPhraseQuery

2018-09-18 Thread Erick Erickson
18, 2018 at 1:56 PM wrote: > > Any suggestions please? > Two main questions: > - how do synonyms get utilized by MultiPhraseQuery? > - how do we get second token "app" applied to the example on > MultiPhraseQuery javadocs page? (and how do we get Terms[] array from &g

Re: MultiPhraseQuery

2018-09-18 Thread baris . kazar
Any suggestions please? Two main questions: - how do synonyms get utilized by MultiPhraseQuery? - how do we get second token "app" applied to the example on MultiPhraseQuery javadocs page? (and how do we get Terms[] array from Terms object?) Now three questions :) i wish the Ja

Re: MultiPhraseQuery

2018-09-18 Thread baris . kazar
;app" then iterating and collecting terms until there is no longer that prefix, // and finally use MultiPhraseQuery.Builder.add(Term[]) to add them. MultiPhraseQuery.Builder.build() returns the fully constructed (and immutable) MultiPhraseQuery. IndexSearcher is = new IndexSearcher(indexRe

MultiPhraseQuery

2018-09-18 Thread baris . kazar
Hi,-  how does MultiPhraseQuery treat synonyms? is the following possible? ... (created index with synonyms and indexReader object has the index) IndexSearcher is = new IndexSearcher(indexReader); MultiPhraseQuery.Builder builder = new MultiPhraseQuery.Builder(); builder.add(new Term("

Re: Crazy increase of MultiPhraseQuery memory usage in Lucene 5 (compared with 3)

2016-10-05 Thread Trejkaz
Thought I would try some thread necromancy here, because nobody replied about this a year ago. Now we're on 5.4.1 and the numbers changed a bit again. Recording best times for each operation. Indexing: 5.723 s SpanQuery: 25.13 s MultiPhraseQuery: (waited 10 minutes and it h

Re: Crazy increase of MultiPhraseQuery memory usage in Lucene 5 (compared with 3)

2015-08-23 Thread Trejkaz
icial test, but benchmarks tend to be like that. Times for Lucene 3.6: Indexing: 3.365 s SpanQuery: 20.48 s MultiPhraseQuery: 9.641 s Times for Lucene 5.2: Indexing: 4.423 s SpanQuery: 31.94 s MultiPhraseQuery: (never completes due to OOME) An aside which is totally a red h

Crazy increase of MultiPhraseQuery memory usage in Lucene 5 (compared with 3)

2015-08-23 Thread Trejkaz
There is a MultiPhraseQuery we use which looks a bit like: MultiPhraseQuery query = new MultiPhraseQuery(); query.add(new Term[] { "first" }); query.add(new Term[] { "second1", "second2", ... }); The actual number of terms in this particular case is

Is it possible to rewrite a MultiPhraseQuery to a SpanQuery?

2014-08-18 Thread Trejkaz
Someone asked if it was possible to do a SpanNearQuery between a TermQuery and a MultiPhraseQuery. Sadly, you can only use SpanNearQuery with other instances of SpanQuery, so we have a gigantic method where we rewrite as many queries as possible to SpanQuery. For instance, TermQuery can trivially

Re: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-04 Thread Michael McCandless
OK, phew :) Thanks for bringing closure... Mike http://blog.mikemccandless.com On Wed, May 4, 2011 at 6:52 AM, Tomislav Poljak wrote: > Hi, > seems there is a custom impl of MultiPhraseQuery used in the system, > which uses (and maybe misuses) Lucene's MultiPhraseQuery that

Re: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-04 Thread Tomislav Poljak
Hi, seems there is a custom impl of MultiPhraseQuery used in the system, which uses (and maybe misuses) Lucene's MultiPhraseQuery that could be the reason of slowdown. I've tried running sample Lucene's MultiPhraseQuery in an infinite while loop printing out times for every 1000

Re: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-03 Thread Michael McCandless
sted running same query list (~3,5 k queries) on the same > MemoryIndex instance and after a while iterations get slower and > slower. Same thing happens when running queries on the same instance > of RAMDir based index holding only one doc. But, if I remove > MultiPhraseQuery type of quer

RE: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-03 Thread Uwe Schindler
the same > MemoryIndex instance and after a while iterations get slower and slower. > Same thing happens when running queries on the same instance of RAMDir > based index holding only one doc. But, if I remove MultiPhraseQuery type of > queries from the query list then speed of execution is

Re: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-03 Thread Tomislav Poljak
le iterations get slower and slower. Same thing happens when running queries on the same instance of RAMDir based index holding only one doc. But, if I remove MultiPhraseQuery type of queries from the query list then speed of execution is the same, meaning execution time for other queries is constant an

Re: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-03 Thread Michael McCandless
he > queries are MultiPhraseQueries.  This is even with with mergeSort being used > in > MultiPhraseQuery. > > Otis > > Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch > Lucene ecosystem search :: http://search-lucene.com/ > > > > - Orig

Re: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-02 Thread Otis Gospodnetic
nger and longer -- if some of the queries are MultiPhraseQueries. This is even with with mergeSort being used in MultiPhraseQuery. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message > From:

Re: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-02 Thread Michael McCandless
/known issues related to the MultiPhraseQuery in Lucene 3.1 > which could lead to this performance drop? > > Tomislav > > - > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional c

RE: MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-02 Thread Uwe Schindler
m: Tomislav Poljak [mailto:tpol...@gmail.com] > Sent: Monday, May 02, 2011 6:01 PM > To: java-user@lucene.apache.org > Subject: MultiPhraseQuery slowing down over time in Lucene 3.1 > > Hi, > after running tests on both MemoryIndex and RAMDirectory based index in > Lucene 3

MultiPhraseQuery slowing down over time in Lucene 3.1

2011-05-02 Thread Tomislav Poljak
related to the MultiPhraseQuery in Lucene 3.1 which could lead to this performance drop? Tomislav - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org

Re: Scoring problem with MultiPhraseQuery?

2010-12-16 Thread Ian Lea
More likely to be a bug in your code rather than in lucene. I suggest you post the smallest possible self-contained program or test case that demonstrates the problem. -- Ian. On Thu, Dec 16, 2010 at 1:56 AM, Mike Cawson wrote: > I'm using MultiPhraseQuery to implement a fuzzy phra

Scoring problem with MultiPhraseQuery?

2010-12-15 Thread Mike Cawson
I'm using MultiPhraseQuery to implement a fuzzy phrase query. E.g. user enters "blue lorry" and I expand 'blue' to 'turquoise', and 'glue' and 'lorry' to 'truck', 'van', 'lory' and 'lorrie'. I ca

Re: MultiPhraseQuery throws ArrayIndexOutOfBounds Exception

2010-07-30 Thread Michael McCandless
Nice catch -- thanks! I will fix. Mike On Fri, Jul 30, 2010 at 11:20 AM, jayendra patil wrote: > Working on the nightly build of solr and lucene - > > MultiPhraseQuery throws ArrayIndexOutOfBounds Exception for the words > defined as synonym

MultiPhraseQuery throws ArrayIndexOutOfBounds Exception

2010-07-30 Thread jayendra patil
Working on the nightly build of solr and lucene - MultiPhraseQuery throws ArrayIndexOutOfBounds Exception for the words defined as synonyms SEVERE: java.lang.ArrayIndexOutOfBoundsException: 5 at org.apache.lucene.search.MultiPhraseQuery$MultiPhraseWeight.scorer(MultiPhraseQuery.java:191

Re: PhraseQuery vs MultiPhraseQuery

2010-05-28 Thread Ahmet Arslan
> Is there a fundamental difference between > > PhraseQuery query = new PhraseQuery(); > query.add(term1, 0); > query.add(term2, 0); > > and > > MultiPhraseQuery query = new MultiPhraseQuery(); > query.add( new Term[] { term1, term2 } ); > > The only di

PhraseQuery vs MultiPhraseQuery

2010-05-28 Thread Emmanuel Bernard
Hello, I am a bit confused by the two. Is there a fundamental difference between PhraseQuery query = new PhraseQuery(); query.add(term1, 0); query.add(term2, 0); and MultiPhraseQuery query = new MultiPhraseQuery(); query.add( new Term[] { term1, term2 } ); The only different I could think of

RE: Exception invoking MultiPhraseQuery

2010-01-12 Thread Woolf, Ross
org Subject: Re: Exception invoking MultiPhraseQuery I'd try running it outside of Eclipse, and/or checking each and every of the many configuration options in Eclipse to see if you have an old jar that Eclipse is using, from jars you've made accessible via the "java build path" window to proj

Re: Exception invoking MultiPhraseQuery

2010-01-12 Thread Erick Erickson
ly, you can look for all the Lucene jars on your machine and delete (or move) any old ones. And if none of this helps, can you post the entire stack trace? HTH Erick On Tue, Jan 12, 2010 at 1:28 PM, Woolf, Ross wrote: > I can't invoke MultiPhraseQuery. It produces the error: >

Exception invoking MultiPhraseQuery

2010-01-12 Thread Woolf, Ross
I can't invoke MultiPhraseQuery. It produces the error: com.sun.jdi.InvocationException occurred invoking method Here is the code: MultiPhraseQuery mpq = new MultiPhraseQuery(); In the eclipse debugger when I try to inspect mpq after instantiating it shows the error. I'm on Lucene

bug in MultiPhraseQuery toString() method, ArrayIndexOutOfBoundsException

2008-04-30 Thread Robert . Hastings
Using Lucene 2.3.0 I'm seeing an ArrayIndexOutOfBoundsException: 0 at line 291 of MultiPhraseQuery. A test should be added for (terms.length == 0). I'm checking to see why the terms array is 0. Bob Hastings

wildcard in phrase query: problem with idf / scoring; QueryParser; MultiPhraseQuery

2006-07-03 Thread W.H. van Atteveldt
'prefix phrase queries', ie "microsoft app*". If I understand the doc and mailing list correctly, there is some 'backend plumbing' for this in the sense of the MultiPhraseQuery, but the QueryParser does not handle this. Is this correct? Will this change in the near future?

Re: MultiPhraseQuery

2006-03-07 Thread Erik Hatcher
On Mar 7, 2006, at 2:35 AM, Eric Jain wrote: Daniel Naber wrote: Please try to add this to MultiPhraseQuery and let us know if it helps: public List getTerms() { return termArrays; } That is indeed all I need (the list wouldn't have to be mutable though). Any chance this cou

Re: MultiPhraseQuery

2006-03-06 Thread Eric Jain
Daniel Naber wrote: Please try to add this to MultiPhraseQuery and let us know if it helps: public List getTerms() { return termArrays; } That is indeed all I need (the list wouldn't have to be mutable though). Any chance this could be committed? Incidentally, would be helpf

Re: MultiPhraseQuery

2006-03-06 Thread Erik Hatcher
On Mar 6, 2006, at 4:43 PM, Daniel Naber wrote: On Sonntag 05 März 2006 19:03, Eric Jain wrote: I need to write a function that copies a MultiPhraseQuery and changes the field the query applies to. Unfortunately the API allows access to neither the contained terms nor the field! The other

Re: MultiPhraseQuery

2006-03-06 Thread Daniel Naber
On Sonntag 05 März 2006 19:03, Eric Jain wrote: > I need to write a function that copies a MultiPhraseQuery and changes > the field the query applies to. Unfortunately the API allows access to > neither the contained terms nor the field! The other query classes I > have so far dealt w

MultiPhraseQuery

2006-03-05 Thread Eric Jain
I need to write a function that copies a MultiPhraseQuery and changes the field the query applies to. Unfortunately the API allows access to neither the contained terms nor the field! The other query classes I have so far dealt with all seem to allow access to the contained query terms