Re: Forcemerge running out of memory

2013-04-11 Thread Wei Wang
Strange. This happens after I added all documents using IndexWriter.addDocument() function. Everything works well at that point. I then call IndexWriter.forceMerge(1), finally IndexWriter.close(true). The "out of memory" problem happens after I called forceMerge(1) but before close(true). If mer

Re: Forcemerge running out of memory

2013-04-11 Thread Robert Muir
merging binarydocvalues doesn't use any RAM, it streams the values from the segments its merging directly to the newly written segment. So if you have this problem, its unrelated to merging: it means you don't have enough RAM to support all the stuff you are putting in these binarydocvalues fields

Forcemerge running out of memory

2013-04-11 Thread Wei Wang
Hi, After finishing indexing, we tried to consolidate all segments using forcemerge, but we continuously get out of memory error even if we increased the memory up to 4GB. Exception in thread "main" java.lang.IllegalStateException: this writer hit an OutOfMemoryError; cannot complete forceMerge