Hi all, I have been looking for information about this and found a few things here and there but nothing very clear on when files are opened and closed by Lucene.
We have an application that uses Lucene quite heavily in the following fashion: there are multiple indexes in use at all times. For each index, a single IndexWriter object is constantly opened (and used to update the index) and a single IndexReader is also constantly opened. This Reader is created using the almost real-time reader of the IndexWriter. The Reader is used to query the index and when the Writer updates an index, the corresponding reader is re-opened and the previous one is discarded when all searches using it are finished. Periodically, the index is optimised if it needs to be (if the writer has been used). This is all working perfectly fine but we've been running into an issue recently when we've add a lot of indexes on one server (more than a 1000 indexes). This led to the "Too many open files" exception, so we increased the number of open files allowed on that server but obviously, it's a temporary solution. So we decided to implement a mechanism that would keep only a maximum number of indexes "open" at all times. When the maximum number is reached, the least recently used indexes are closed (closing the IndexWriter and the IndexReader), and if these indexes are needed again in the future, they are "re-opened". This is functioning fine as well, as we can see ".lock" files appearing and disappearing in the index directories, but handles on the segment files are still in use. So my question is when are these file handles discarded? I was under the impression from reading the documentation that closing an IndexReader was enough to close the files associated with that reader but we went even further by closing the IndexWriter as well but to no avail. If it should work that way, it means we have a bug somewhere with readers not being closed properly but I couldn't find one last time I checked. Also, do you think it is the right way to go around the "too many open files" issue? Can you recommend using another method? Thanks for any help. JB Reure
