Re: search performance enhancement

2005-08-19 Thread Paul Elschot
On Friday 19 August 2005 18:09, John Wang wrote: > Hi Paul: > > Thanks for the pointer. > >How would I extend from the patch you submitted to filter out > more documents not using a Filter. e.g. > > have a class to skip documents based on a docID: boolean > isValid(int docID)

Re: Most efficient "all documents" query?

2005-08-19 Thread Daniel Naber
On Friday 19 August 2005 17:04, Jeff Davis wrote: > I am using Lucene to index system and application log files.  When a > user first goes to the Log Viewer, I want them to see all log > messages.  From there, they can choose to search for something more > specific. This might be what you are loo

Use of ThreadLocal in TermInfosReader

2005-08-19 Thread Lee Turner
Hi We am currently investigating an issue I have with our application around its memory consumption. We are running our web application under load with 20 users and 1GB heap size and at around 7 hours it runs out of memory. We are using Lucene 1.4.3 and JDK 1.4.2_06. We are working with a

Re: search performance enhancement

2005-08-19 Thread John Wang
Hi Paul: Thanks for the pointer. How would I extend from the patch you submitted to filter out more documents not using a Filter. e.g. have a class to skip documents based on a docID: boolean isValid(int docID) My problem is I want to discard documents at query time wit

Most efficient "all documents" query?

2005-08-19 Thread Jeff Davis
I am using Lucene to index system and application log files. When a user first goes to the Log Viewer, I want them to see all log messages. From there, they can choose to search for something more specific. Since the "all" query will be by far the most frequently used one, I'd like to make it as

Re: Case-sensitive search

2005-08-19 Thread tareque
> > On Aug 18, 2005, at 6:22 PM, [EMAIL PROTECTED] wrote: > >>> On Thu, 2005-08-18 at 17:16, [EMAIL PROTECTED] wrote: >>> Thanks again! The analyzer is working now. But seems like actually the QueryParser I am using is probably converting the queries to lowercase first. Is

QueryParser issue

2005-08-19 Thread Stanislav Jordanov
Recently I found that it makes difference whether your query looks like: A AND NOT B or it is A AND (NOT B) Some research using Luke showed that in the first case the query is rewritten as: +A -B while the second is rewritten as +A +(-B) after the usual "wtf?" I've recalled that there was a dicu

Re: OutOfMemory error when searching

2005-08-19 Thread Fredrik
Sorry about the previous stack trace -- that was a wild goose-chase. Here is the real culprit: at org.apache.lucene.index.SegmentTermEnum.growBuffer(SegmentTermEnum.java(Compiled Code)) at org.apache.lucene.index.SegmentTermEnum.readTerm(SegmentTermEnum.java(Compiled Code)) at org.apache.lucen

Re: DEFAULT_OPERATOR_AND

2005-08-19 Thread Erik Hatcher
On Aug 19, 2005, at 3:12 AM, Karthik N S wrote: With refrence to the code as below public static Query Sparse(String texts, String[] fields, Analyzer analyzer) { Query bQuery = null; try { for (int i = 0; i < fields.length; i++) { QueryParser qp = new QueryParse

RE: Case-sensitive search

2005-08-19 Thread Peter Gelderbloem
I don't really need this at the moment, but need to know: Is it conceivable to make a configurable analyzer with dynamically added filters? Peter -Original Message- From: Erik Hatcher [mailto:[EMAIL PROTECTED] Sent: 19 August 2005 01:50 To: java-user@lucene.apache.org Subject: Re: Case-

Re: OutOfMemory error when searching

2005-08-19 Thread Volodymyr Bychkoviak
Hi, IndexReader class has static method getCurrentVersion. you can check index version and if was changed since last time then reopen IndexSearcher. Peter Gelderbloem wrote: Hi, I foresee that I may run into similar problems as I am opening and closing IndexSearchers all over the place. Unfo

Re: OutOfMemoryError on addIndexes()

2005-08-19 Thread Tony Schwartz
Aha, it's not initially clear, but after looking at it more closely, I see how it works now. This is very good to know. Tony Schwartz [EMAIL PROTECTED] > Tony Schwartz wrote: >> What about the TermInfosReader class? It appears to read the entire term >> set for the >> segment into 3 arrays

RE: OutOfMemory error when searching

2005-08-19 Thread Peter Gelderbloem
Hi, I foresee that I may run into similar problems as I am opening and closing IndexSearchers all over the place. Unfortunately I do not see a way to keep the IndexSearcher open as indexing occurs at random times in my system and the searches need to be current. Any suggestions on how to reuse th

Re: Token Filter question

2005-08-19 Thread Paul Elschot
On Thursday 18 August 2005 21:51, Dan Armbrust wrote: > I am implementing a filter that will remove certain characters from the > tokens - thing like '(', etc - but the chars to be removed will be > customizable. > > This is what I have come up with - but it doesn't seem very efficient. > Is t

RE: DEFAULT_OPERATOR_AND

2005-08-19 Thread Karthik N S
Hi Erik Apologies... With refrence to the code as below public static Query Sparse(String texts, String[] fields, Analyzer analyzer) { Query bQuery = null; try { for (int i = 0; i < fields.length; i++) { QueryParser qp = new Que