Numeric Range Filter - bug or documentation oversight

2010-03-05 Thread Scott Smith
I've been updating from 2.4.2 to 3.0.1. I had a number of issues (The Version object in the analyzers was an "interesting" addition-I guess I don't understand the use case for them. I understand what it says; I was just surprised and it caused me some problems since I create analyzers with reflect

Re: OutOfMemoryError

2010-03-05 Thread Otis Gospodnetic
Maybe it's not a leak, Monique. :) If you use sorting in Lucene, then the FieldCache object will keep some data permanently in memory, for example. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Hadoop ecosystem search :: http://search-hadoop.com/ - Original Message -

RE: FastVectorHighlighter truncated queries

2010-03-05 Thread halbtuerderschwarze
Thats the solution. Thanks a lot. -- View this message in context: http://old.nabble.com/FastVectorHighlighter-truncated-queries-tp27709797p27799057.html Sent from the Lucene - Java Users mailing list archive at Nabble.com. -

OutOfMemoryError

2010-03-05 Thread Monique Monteiro
Hi all, I’m new to Lucene and I’m evaluating it in a web application which looks up strings in a huge index – the index file contains 32GB. I keep a reference to a Searcher object during the application’s lifetime, but this object has strong memory requirements and keeps memory consumption aro

RE: FastVectorHighlighter truncated queries

2010-03-05 Thread Digy
queryParser.SetMultiTermRewriteMethod(MultiTermQuery.SCORING_BOOLEAN_QUERY_R EWRITE); DIGY -Original Message- From: halbtuerderschwarze [mailto:halbtuerderschwa...@web.de] Sent: Friday, March 05, 2010 10:56 AM To: java-user@lucene.apache.org Subject: RE: FastVectorHighlighter truncated

Re: recovering payload from fields

2010-03-05 Thread Grant Ingersoll
It's not implemented, but http://issues.apache.org/jira/browse/LUCENE-1888 is how I would solve it. It probably isn't that hard to implement, actually. A patch would be great. Happy to review one. On Feb 27, 2010, at 5:29 PM, Christopher Condit wrote: >> It sounds like you need to iterate t

Re: recovering payload from fields

2010-03-05 Thread Christopher Tignor
What I'd ideally like to do is to take SpanQuery, loop over the PayloadSpans returned from SpanQuery.getPayloadSpans() and store all PayloadSpans for a given document in a Map by their doc id. Then later after deciding in memory which documents I need, load the Payload data for just those PayloadS

Re: File descriptor leak in ParallelReader.reopen()

2010-03-05 Thread Justin
Out of curiosity, I downloaded 2.4.1 and made the necessary source code modifications (attached). There used to be some sort of file descriptor cleanup. With the explicit close, the descriptor count stayed under 100. Without the explicit close, the count hit peaks around 3000 by the time 50,0

Re: how to use DuplicateFilter to get unique documents based on a fieldName

2010-03-05 Thread ani...@ekkitab
Ok sorry for not explaining my problem clearly earlier. We have around 5 fields in each document. ID, ISBN, author, title and the category which this book falls under. ( You are right about point 3, we are indeed storing multiple genre against the book, which means 1 book 1 doc.) doc.add(new Fie

Re: SpanQueries in Luke

2010-03-05 Thread mark harwood
>>The lack of standardized metadata is an issue, of course - we could start experimenting with this in Luke, to see whether we can squeeze a subset of Solr schema there. Actually, an "AnalyzerFactory" interface in Luke might provide the abstraction which would allow Solr, my proprietary metadata

Re: SpanQueries in Luke

2010-03-05 Thread Andrzej Bialecki
On 2010-03-05 11:22, mark harwood wrote: I'll commit the current mostly-working state today, you can take a look OK. However I think this XMLQueryParser addition will only resurface a long-standing issue with Luke and Lucene in general. This query parser works best on multiple fields (e.g. fre

Re: SpanQueries in Luke

2010-03-05 Thread mark harwood
>>I'll commit the current mostly-working state today, you can take a look OK. However I think this XMLQueryParser addition will only resurface a long-standing issue with Luke and Lucene in general. This query parser works best on multiple fields (e.g. free-text tags and on structured fields).

Re: SpanQueries in Luke

2010-03-05 Thread Andrzej Bialecki
On 2010-03-05 10:47, mark harwood wrote: No, this simply means that you will be able to use the xml-query-parser instead of the regular one Not sure exactly what you have in mind for an editor, Andrzej but there is an opportunity to do something smart here for little effort. The XMLQueryPa

Re: DisjunctionMaxQuery and custom Sort

2010-03-05 Thread gabriele renzi
On Fri, Mar 5, 2010 at 10:42 AM, Ian Lea wrote: > From the javadocs for the search with sort method it appears you are calling: > > NOTE: this does not compute scores by default; use > IndexSearcher.setDefaultFieldSortScoring(boolean, boolean) to enable > scoring. > > > Sounds like you need to cal

Re: SpanQueries in Luke

2010-03-05 Thread mark harwood
>>No, this simply means that you will be able to use the xml-query-parser >>instead of the regular one Not sure exactly what you have in mind for an editor, Andrzej but there is an opportunity to do something smart here for little effort. The XMLQueryParser comes with a DTD which means you ca

Re: DisjunctionMaxQuery and custom Sort

2010-03-05 Thread Ian Lea
>From the javadocs for the search with sort method it appears you are calling: NOTE: this does not compute scores by default; use IndexSearcher.setDefaultFieldSortScoring(boolean, boolean) to enable scoring. Sounds like you need to call that with at least the first arg set to true. -- Ian. O

Re: how to use DuplicateFilter to get unique documents based on a fieldName

2010-03-05 Thread Anshum
Hi Anish, So am I getting something wrong here? You said "I have created a search index on book Id , title ,and author from a database of books which fall under various categories." so those are 3 fields, right? 1. How do you filter the doc types (as in the genres) at search time? Do you even need

DisjunctionMaxQuery and custom Sort

2010-03-05 Thread gabriele renzi
Hi everyone, In our search app we'd perform searches with a DisjunctionMaxQuery with code like searcher.search(query, hits) Now we'd like to add a custom sub-sort so that when the search returns two documents with the same score they will be ordered by a custom field (this is not a boost for t

RE: FastVectorHighlighter truncated queries

2010-03-05 Thread halbtuerderschwarze
Thank you very much. You are right, queries like aa* really work fine, even in Lucene 3.0.1 :) But I still have problems getting fragments for queries like aa* AND *ac (parser.setAllowLeadingWildcard(true); is set). -- View this message in context: http://old.nabble.com/FastVectorHighlighter-tr