Hi, I posted a while ago about sudden FileNotFoundExceptions. In a nutshell: my Lucene index went corrupt after a couple of days under heavy load on a Linux server with missing segment files.
The problem kept occuring, but I haven't found the cause. I couldn't reproduce it with a simulated load on my development machine or the test server. Here's a sketch of the setup: there were 14 indexes that received simultaneous updates and search queries. Each index had own thread for updates (so the index wasn;t updated concurrently), but allowed for reading from other threads. Updating consisted of deleting on index intry and adding a new one. On average I would say there was one update request and one search request per second. So, this turned out not to be stable! After a while (a day or so, or two), any index would get corrupted, because a segment would disappear. I circumvented the problem using seperate indexes for searching and updating. The search index is copied every ten minutes to accomodate new search requests. The problem seems to have gone away. So here's my tip: if you have continuous updates and a lot of search requests, use different indexes for searching and for updating! Cheers, Hes.
