Hi, I foresee that I may run into similar problems as I am opening and closing IndexSearchers all over the place. Unfortunately I do not see a way to keep the IndexSearcher open as indexing occurs at random times in my system and the searches need to be current. Any suggestions on how to reuse the IndexSearcher in this situation?
Peter -----Original Message----- From: Doug Cutting [mailto:[EMAIL PROTECTED] Sent: 18 August 2005 19:09 To: java-user@lucene.apache.org Subject: Re: OutOfMemory error when searching Fredrik wrote: > Opening the index with Luke, I can see the following: > Number of fields: 17 > Number of documents: 1165726 > Number of terms: 6721726 > > The size of the index is approx 5,3 GB. > Lucene version is 1.4.3. > > The index contains Norwegian terms, but lots of inline HTML, etc > is probably increasing the index term count (should 'wash' these > unwanted terms away when indexing documents). The analysis below > shows that TermInfosReader.java:132 -> get() is trying to allocate > a huge memory slab. [ ... ] > 'need 532676624 bytes' means that something is allocating a 500Mb slab > of memory. Lucene will try to allocate an array of 6721726/128 ~= 50k terms. The array alone will require a 200kB "slab" and the terms perhaps 1MB or more. But not 500MB. So I think something else is the culprit here. Have you tried inserting print statements at the suspected allocations, to see how big the arrays actually are? Are you perhaps creating a new IndexSearcher per search, rather than reusing a single IndexSearcher? 1MB per query could quickly exhaust RAM if the GC can't keep up. Doug --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]