When you say "getting a reader of the writer" do you mean writer.getReader()? Ie the new near real-time API in 2.9?
For that API (an in general whenever you open a reader), you must close it. I think all your files is because you're not closing your old readers. Reopening readers during optimize is fine, if you close the old reader each time. It will possibly tie up more transient disk usage than had you reopened at the end of optimize, but if you have plenty of disk space it shouldn't be a problem. Mike On Mon, Nov 23, 2009 at 3:20 PM, vsevel <v.se...@lombardodier.com> wrote: > > Hi, I am using lucene 2.9.1 to index a continuous flow of events. My server > keeps an index writer open at all time and write events as groups of a few > hundred followed by a commit. While writing, users invoke my server to > perform searches. Once a day I optimize the index, while writes happens and > searches may happen. I adopted the following strategy: > > for every search I open a new IndexSearcher of the reader of the writer. I > execute the search, fetch the documents and finally close the searcher. > Specifically, I never close the reader, nor the writer. > > Q: is that a reasonnable strategy? > > I found out that my 40Gb index grew up to 200Gb while the number of docs > stayed put at 30 millions. I am suspecting that a search during the optimize > caused this situation, as described in the index writer javadoc (about > refreshing readers during an optimize). > > Q: is that the likely cause? is getting a reader of the writer just as "bad" > as refreshing a reader during an optimize? how can I avoid this behavior? > should I just deny searches while optimizing? > > question on the side: is there any way to interrupt a search that takes too > long? for instance by setting a boolean from another thread on the searcher > currently performing the search. > > thanks, > vincent > -- > View this message in context: > http://old.nabble.com/Searching-while-optimizing-tp26485138p26485138.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
