hi Mike , at the peak there are 500 live threads going through Lucune (not all of them at the same time , tomcat thread pool uses round robin ) ,regarding the Directory impl we are using RAMDirectory. the object that takes most of the heap is the "hardRefs" WeakHashMap class member in the CloseableThreadLocal class . the size of the map is 500 , with one entry for each thread that went through Lucune.
when in run the test with only one thread the initial RAM did not grow much beyond the initial 30MB . when looking at the code i can see that the references to the hardRefs map will be cleared when : 1.a thread which searched Lucune is no longer alive , so its corresponding record in the map will be cleared either by the GC or by the purge() method . 2.the close() method is called which "should only be called when all threads are done using the instance" . what happens to threads that go through Lucune and stay alive for a very long times ? will they always be a reference key in the hardRefs map for them ? if so does the value for the corresponding record in the map will be overwritten each time this thread makes a new search ? or will it be some how accumulated ? : value ---- org.apache.lucene.index.TermInfosReader$ThreadResources ---> termInfoCache |org.apache.lucene.util.cache.SimpleLRUCache termEnum |org.apache.lucene.index.SegmentTermEnum i have reduced the size of the thread pool , and changed to a more aggressive thread closing policy ,threads are now reduced when they are not needed (before that the thread pool would reach its max size and stayed there), this should kick in the the purge method and reclaim memory from the closed threads . ill give the system a few hours to run and update on the results . Alon On Thu, Jan 3, 2013 at 2:58 AM, Michael McCandless < luc...@mikemccandless.com> wrote: > then see if RAM doesn't grow much beyond that initial 30 MB?
