Re: JAVA JVM Question

2006-12-22 Thread Otis Gospodnetic
The latest Java (6) does produce a stack trace when am OOM error happens. It doesn't necessarily point to the source of the leak, though, just at the unfortunate thing that needed the non-existent memory to do its work. Otis - Original Message From: Simon Willnauer <[EMAIL PROTECTED]>

Re: JAVA JVM Question

2006-12-22 Thread Simon Willnauer
Do you have a stack trace by any chance, well OOM do not create very meanful stacktraces but it could help to figure out which error occures. PermGenSpace belogs to the java heap btw. The JBoss should actually be able to deal with your redeployment but in live/production a restart is always requie

Re: java.lang.NoClassDefFoundError: com_cenqua_clover/g

2006-12-22 Thread Grant Ingersoll
This is a bug that just got introduced by the new nightly build process. See http://issues.apache.org/jira/browse/LUCENE-708 I will work to fix before tonight's build. -Grant On Dec 22, 2006, at 2:36 PM, Suman Ghosh wrote: I am trying to use Lucene nightly build lucene-2006-12-22.tar.gz.

java.lang.NoClassDefFoundError: com_cenqua_clover/g

2006-12-22 Thread Suman Ghosh
I am trying to use Lucene nightly build lucene-2006-12-22.tar.gz. When my code tries to execute: IndexSearcher searcher = new IndexSearcher("/home/lucene/index"); I am receiving the following error: java.lang.NoClassDefFoundError: com_cenqua_clover/g java.lang.NoClassDefFoundError: com_cenqua_

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-22 Thread Yonik Seeley
On 12/22/06, Mark Miller <[EMAIL PROTECTED]> wrote: So the first time you do a sort, the fieldcache is loaded up that stores the term to sort on for each document id. Right. The actual sorting appears to happen just like with relevancy score sortingusing a priority queue that is loaded as

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-22 Thread Mark Miller
I am no expert, but as I gloss over the code this is what I see happens for a sort (sometimes the less experienced has to get it wrong before an expert will jump in with some good info *hint to experts*): The field cache caches pairs. When you sort on a field you don't want to have to extrac

Re: IOException - The handle is invalid

2006-12-22 Thread Michael McCandless
Antony Bowesman wrote: Hi, I'm running load tests with Lucene 2.0, SUN's JDK 6 on Windows XP2, dual core CPU. I have 8 worker threads adding a few hundred K documents, split between two Lucene indexes, I've started getting java.io.IOException: The handle is invalid in places like java.io.R

Re: Merge Index Filling up Disk Space

2006-12-22 Thread Michael McCandless
Harini Raghavan wrote: Thank you for the response. I don't have readers open on the index, but while the optimize/merge was running I was searching on the index. Would that make any difference? You're welcome! Right, a searcher opens an IndexReader. So this means you should see peak @ 3X th

Re: Merge Index Filling up Disk Space

2006-12-22 Thread Mark Miller
A Searcher uses a Reader to read the index for searching. - Mark Harini Raghavan wrote: Hi Mike, Thank you for the response. I don't have readers open on the index, but while the optimize/merge was running I was searching on the index. Would that make any difference? Also after the optimizin

RE: First search is slow after updating index .. subsequent searches very fast

2006-12-22 Thread Bryan Dotzour
Thanks for that tidbit Mark. I was just looking through the LIA book and stumbled across this sentence under the "5.1.9 Performance effect of sorting" section. It says: "[When sorting by a String type] each unique term is also cached for each document. Only the actual fields used for sorting are

Re: Merge Index Filling up Disk Space

2006-12-22 Thread Harini Raghavan
Hi Mike, Thank you for the response. I don't have readers open on the index, but while the optimize/merge was running I was searching on the index. Would that make any difference? Also after the optimizing the index I had some .tmp files which were > 10G and did not get merged. Could that also

Re: Rebuilding index on a regular basis

2006-12-22 Thread Erick Erickson
Adam: I think you're worrying about the wrong thing. There is no "period of unserviceability" to worry about in closing/reopening a searcher. If, by saying "searcher", you mean the Lucene IndexSearcher/Reader. If you're talking about shutting down your service, that's another story. What you *do

Re: French stemmer problem

2006-12-22 Thread Patrick Turcotte
Hi Renaud, Maybe you should take a look at the Morphalou project ( http://actarus.atilf.fr/lexiques/morphalou/) it is a database of lemma and forms in French. You could extract the data and create a synonym index or something. Don't hesitate to contact me off list (and in French if needed) for

Re: French stemmer problem

2006-12-22 Thread Mark Miller
Non of the stemmers always stem to a valid word. It is not important as you should be stemming the query as well. The only thing that is important is that each word always stems to the same base. Many English words do not stem to real English words with the English stemmer either. Renaud Paqua

RE: French stemmer problem

2006-12-22 Thread Samir Abdou
Hi, Take a look to http://www.unine.ch/info/clef where you'll find valuable resources for many languages including French. Samir -Message d'origine- De : Renaud Paquay [mailto:[EMAIL PROTECTED] Envoyé : vendredi, 22. décembre 2006 10:54 À : java-user@lucene.apache.org Objet : Frenc

French stemmer problem

2006-12-22 Thread Renaud Paquay
Hello, Does anyone know about a modified version of the French Stemmer ? This one has too many bad results. For example, if I use the word : "ours" (bear) The stemmer stemm it into "our".which doesn't exist in French. If I have some words like "L'insepecteur" the index process using the stemme