Hi Mike, I just changed my test-code to run in an indefinite loop over the database to index everything. Set the jvm to 120MB heap size, all other parameters as before. I got an OOError just as before - so I would say there is a leak somewhere.
Here is the histogram. Heap Histogram All Classes (excluding platform) Class Instance Count Total Size class [B 1809102 41992326 class [C 200610 26877068 class [[B 46117 9473872 class java.lang.String 198629 3178064 class org.apache.lucene.index.FreqProxTermsWriter$PostingList 100927 2825956 class [Ljava.util.HashMap$Entry; 11329 2494312 class java.util.HashMap$Entry 132578 2121248 class [I 5186 2097300 So far I had no success in pinpointing those binary arrays, I will need some more time for this. Stefan -----Ursprüngliche Nachricht----- Von: Michael McCandless [mailto:luc...@mikemccandless.com] Gesendet: Mi 24.06.2009 17:50 An: java-user@lucene.apache.org Betreff: Re: OutOfMemoryError using IndexWriter On Wed, Jun 24, 2009 at 10:18 AM, stefan<ste...@intermediate.de> wrote: > > Hi, > > >>OK so this means it's not a leak, and instead it's just that stuff is >>consuming more RAM than expected. > Or that my test db is smaller than the production db which is indeed the case. But a "leak" would keep leaking over time, right? Ie even a 1 GB heap on your test db should eventually throw OOME if there's really a leak. > Please explain those buffered deletes in a few more details. Are you calling updateDocument (which deletes then adds)? Deletes (the Term or Query you pass to updateDocument or deleteDocuments) are buffered in a HashMap and then that buffer is materialized into actual deleted doc IDs when IndexWriter decides to do so. I think IndexWriter isn't properly flushing the deletes when they use too much RAM. Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
--------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
