Re: old fashioned....."Too many open files"!

2012-05-17 Thread findbestopensource
Post complete code. You are not closing the objects (IndexWriter / Index Searcher) properly. Regards Aditya www.findbestopensource.com On Fri, May 18, 2012 at 6:51 AM, Michel Blase wrote: > Hi all, > > I have few problems Indexing. I keep hitting "Too many open files". It > seems like Lucene i

Re: Search Ranking

2012-05-17 Thread Ivan Brusic
If you read the explain output, you can see where the scores are different. One difference with a noticeable affect is: 1.0 = tf(termFreq(searchText:fred)=1) 0.5 = fieldNorm(field=searchText, doc=1) vs. 1.4142135 = tf(termFreq(searchText:fred)=2) 0.375 = fieldNorm(field=searchText, doc=0) As pred

Re: NullPointerException using IndexReader.termDocs when there are no matches

2012-05-17 Thread Michael McCandless
I think you need to pay attention to what td.next() returned; I suspect in your case it returned false which means you cannot use any of its APIs (.doc(), .freq(), etc.) after that. Mike McCandless http://blog.mikemccandless.com On Thu, May 17, 2012 at 5:52 PM, Edward W. Rouse wrote: > Lucene 3

NullPointerException using IndexReader.termDocs when there are no matches

2012-05-17 Thread Edward W. Rouse
Lucene 3.6, java 1.6 I get the following: java.lang.NullPointerException at org.apache.lucene.index.DirectoryReader$MultiTermDocs.doc(DirectoryReader.ja va:1179) when running this code: IndexReader reader = this.getReader(index); int d = -1; TermDocs td = reader.termDocs(this.createIdTerm(id));

Re: Store a query in a database for later use

2012-05-17 Thread Trejkaz
On Fri, May 18, 2012 at 6:23 AM, Jamie Johnson wrote: > I think you want to have a look at the QueryParser classes.  Not sure > which you're using to start with but probably the default QueryParser > should suffice. There are (at least) two catches though: 1. The semantics of a QueryParser might

Sort runs out of memory

2012-05-17 Thread Robert Bart
Hi all, I am running Lucene 3.6 in a system that indexes about 4 billion documents across several indexes, and I'm hoping to get documents in order of a certain NumericField. I've tried using Lucene's Sort implementation, but it looks like it tries to do the entire sort in memory by allocating a

Re: Store a query in a database for later use

2012-05-17 Thread Jamie Johnson
I think you want to have a look at the QueryParser classes. Not sure which you're using to start with but probably the default QueryParser should suffice. On Thu, May 17, 2012 at 3:53 PM, Stefan Undorf wrote: > Hi, > > I want to store a query for later use in a database, like: > > 1. queryToPers

Re: Approches/semantics for arbitrarily combining boolean and proximity search operators?

2012-05-17 Thread Alan Woodward
You're right, those cases won't be covered, and probably can't be without some hacking at the NearSpans* classes. The other niggle I've found is that it doesn't play well with highlighting - you get the entire span highlighted, rather than the individual terms within it. For NOT WITHIN queries

Store a query in a database for later use

2012-05-17 Thread Stefan Undorf
Hi, I want to store a query for later use in a database, like: 1. queryToPersist = booleanQuery.toString(); 2. store it to the db, go fishing, retrieve it 3. Query query = Query.parseString(queryToPersist) The method Query.parseString does not exist. Is there a way to do something similar?

Re: Approches/semantics for arbitrarily combining boolean and proximity search operators?

2012-05-17 Thread Chris Harris
First impression is, that's a reasonably clever way to get the user intent basically right without having to add a new SpanQuery. Have you come up with any edge cases where it could do something unexpected? So far I've thought of one, though you could argue it has more to do with the "minimum/lazy

RE: IndexWriter.deleteDocuments(Term) only deletes one at a time

2012-05-17 Thread Edward W. Rouse
Never mind, turns out the Term isn't matching for some as yet unknown reason and the feedback in my logs was misleading. Sorry for the noise. > -Original Message- > From: Edward W. Rouse [mailto:ero...@comsquared.com] > Sent: Thursday, May 17, 2012 10:19 AM > To: java-user@lucene.apache.or

IndexWriter.deleteDocuments(Term) only deletes one at a time

2012-05-17 Thread Edward W. Rouse
Below is the relevant portion of the code were this occurs. I have a test index with 100 Documents that all have 1 Term that is the same across all Documents. When I run this code using that Term and the common value, it only deletes one document. Every time I rerun the code with the same values it

Re: Optional Terms

2012-05-17 Thread Meeraj Kunnumpurath
Thanks Ian Sent from my iPhone On 17 May 2012, at 09:29, Ian Lea wrote: >> Document doc3 = new Document(); >> doc2.add(new Field("searchText", "LMN Takeaway", Field.Store.YES, > > doc2 != doc3. > > Boosting by number of occurrences tends to happen automatically. See > IndexSearcher.explain()

Re: Optional Terms

2012-05-17 Thread Ian Lea
> Document doc3 = new Document(); > doc2.add(new Field("searchText", "LMN Takeaway", Field.Store.YES, doc2 != doc3. Boosting by number of occurrences tends to happen automatically. See IndexSearcher.explain() as I think someone already suggested. See also javadocs for org.apache.lucene.search.S

Re: Approches/semantics for arbitrarily combining boolean and proximity search operators?

2012-05-17 Thread Alan Woodward
I've just had to implement exactly this - the solution I came up with was to translate: A w/5 (B and C) -> SpanNear(A, spanNear(A, B, 5), spanNear(A, C, 5), 0) A w/5 (B or C) -> OR(spanNear(A, B, 5), spanNear(A, C, 5)) More complex queries (such as (A AND B) w/5 (C AND D)) are dealt with by app