Ajay, I've posted a few times on OOM issues. Here is one thread.
http://mail-archives.apache.org/mod_mbox//lucene-java-user/200909.mbox/% 3c5b20def02611534db08854076ce825d803626...@sc1exc2.corp.emainc.com%3e I'll try and get some more links to you from some other threads I started for OOM issues and Lucene. I've not solved this problem yet. Please let me know if you come up with a solution. Hope this helps. Paul -----Original Message----- From: java-user-return-45254-paul.b.murdoch=saic....@lucene.apache.org [mailto:java-user-return-45254-paul.b.murdoch=saic....@lucene.apache.org ] On Behalf Of ajay_gupta Sent: Tuesday, March 02, 2010 8:28 AM To: java-user@lucene.apache.org Subject: Lucene Indexing out of memory Hi, It might be general question though but I couldn't find the answer yet. I have around 90k documents sizing around 350 MB. Each document contains a record which has some text content. For each word in this text I want to store context for that word and index it so I am reading each document and for each word in that document I am appending fixed number of surrounding words. To do that first I search in existing indices if this word already exist and if it is then I get the content and append the new context and update the document. In case no context exist I create a document with fields "word" and "context" and add these two fields with values as word value and context value. I tried this in RAM but after certain no of docs it gave out of memory error so I thought to use FSDirectory method but surprisingly after 70k documents it also gave OOM error. I have enough disk space but still I am getting this error.I am not sure even for disk based indexing why its giving this error. I thought disk based indexing will be slow but atleast it will be scalable. Could someone suggest what could be the issue ? Thanks Ajay -- View this message in context: http://old.nabble.com/Lucene-Indexing-out-of-memory-tp27755872p27755872. html Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
