Gusenbauer Stefan wrote:
A few weeks before I had a similar problem too. I will write my problem
and the solution for it:
I'm indexing docs and every parsed document is stored in an ArrayList.
This solution worked for little directories with a little number of
files in it but when the things are growing you're in trouble.
My solution was whenever I will run out of memory I will "save" the
documents. I open the indexwriter and write every document from the
arraylist to the index. Then I set the arraylist and some other stuff =
null and try to invoke the garbage collector. Then I do some
reinitializing and continue indexing.
Looks easy but it wasn't. How do I check if i will run out of memory?
Runtimeclass and its methods for getting information about the free
memory were very unreliable.
Therefore I changed to Java 1.5 and implemented a memorynotification
listener which is support by the java.lang.management package. There you
can adjust a threshold when you should be informed. After the
notification I perform a "save".
Hope this will help you
Stefan
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Thank you Stefan,
unfortunately, our situation is a bit different - we are not caching
parsed docs in any way.
When a document is parsed it is indexed immediately.
So in our case it is not the accumulation of documents waiting to be
indexed that causes the OutOfMemory exception.
I believe it is a "pure lucene" issue - just as at some point when the
next doc is added to the index, and this causes (perhaps)
the merging of segments, the memory consumption raises drastically.
Regards
Stanislav
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
