I suspect you're already aware of this, but I've overlooked the obvious so many times I thought I'd mention it...
A classic mistake is to assign a reader with reopen and not close the old reader, see: http://lucene.apache.org/java/3_0_1/api/core/org/apache/lucene/index/IndexReader.html#reopen() <http://lucene.apache.org/java/3_0_1/api/core/org/apache/lucene/index/IndexReader.html#reopen()> Erick. On Tue, Apr 5, 2011 at 7:59 PM, Jean-Baptiste Reure <jbre...@weborganic.com>wrote: > We are using version 3.0.3. So you can confirm that closing the writer (and > the reader created from that writer) should be enough to release the file > handles? > If that is the case, that means our application has a bug somewhere that i > need to track down. > Thanks, > JB. > > On 5 April 2011 19:48, Ian Lea <ian....@gmail.com> wrote: > > > Which version of lucene are you using? Something changed in the 3.x > > release, and maybe 2.9.x, in the way that old file handles are closed. > > Previously it wasn't always necessary to explicitly close everything, > > now it is. > > > > Your usage sounds fine to me. When I've hit too many open files when > > using 3.0.x it has always been because something, somewhere, wasn't > > being closed. > > > > > > -- > > Ian. > > > > > > On Tue, Apr 5, 2011 at 8:49 AM, Jean-Baptiste Reure > > <jbre...@weborganic.com> wrote: > > > Hi all, > > > > > > I have been looking for information about this and found a few things > > here > > > and there but nothing very clear on when files are opened and closed by > > > Lucene. > > > > > > We have an application that uses Lucene quite heavily in the following > > > fashion: there are multiple indexes in use at all times. For each > index, > > a > > > single IndexWriter object is constantly opened (and used to update the > > > index) and a single IndexReader is also constantly opened. This Reader > is > > > created using the almost real-time reader of the IndexWriter. The > Reader > > is > > > used to query the index and when the Writer updates an index, the > > > corresponding reader is re-opened and the previous one is discarded > when > > all > > > searches using it are finished. Periodically, the index is optimised if > > it > > > needs to be (if the writer has been used). > > > > > > This is all working perfectly fine but we've been running into an issue > > > recently when we've add a lot of indexes on one server (more than a > 1000 > > > indexes). This led to the "Too many open files" exception, so we > > increased > > > the number of open files allowed on that server but obviously, it's a > > > temporary solution. > > > > > > So we decided to implement a mechanism that would keep only a maximum > > number > > > of indexes "open" at all times. When the maximum number is reached, the > > > least recently used indexes are closed (closing the IndexWriter and the > > > IndexReader), and if these indexes are needed again in the future, they > > are > > > "re-opened". > > > This is functioning fine as well, as we can see ".lock" files appearing > > and > > > disappearing in the index directories, but handles on the segment files > > are > > > still in use. > > > > > > So my question is when are these file handles discarded? I was under > the > > > impression from reading the documentation that closing an IndexReader > was > > > enough to close the files associated with that reader but we went even > > > further by closing the IndexWriter as well but to no avail. If it > should > > > work that way, it means we have a bug somewhere with readers not being > > > closed properly but I couldn't find one last time I checked. > > > > > > Also, do you think it is the right way to go around the "too many open > > > files" issue? Can you recommend using another method? > > > > > > Thanks for any help. > > > > > > JB Reure > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > >
