Hello, One of our Lucene indexes has started misbehaving on indexWriter.close and I'm searching for ideas about what may have happened and how to fix it.
Here's our scenario: - We have seven Lucene indexes that contain different sets of data from a web application are indexed for searching by end users - A java service runs twice a minute to pull changes from SQL DB queue tables and update the relevant Lucene index(es) - The two largest indexes (3.4GB and 3.8GB in size with 8 million and 6 million records, respectively) contain similar sets of data, but are structured differently for different consumption (e.g. one has an All field for general purpose searching, the other does not; one has numeric fields for ranges, the other does not, etc.) - We expunge deletes from our indexes twice per day - A couple of weeks ago, one of the two large indexes became very slow to close after each round of changes is applied by our indexing service. Specifically, all of our indexes usually close in no more than 200 milliseconds, but this one index is now taking 6 to 8 seconds to close with every single pass (and is leading to delays which are affecting end users). Questions from my attempts to troubleshoot the problem: - Has anyone else seen behavior similar to this? What did you do to resolve it? - Does the size of an index or its documents (record count, disk size, avg document size, max document size, etc.) have any correlation to the length of time it takes to close an index? - We are not currently optimizing any of our indexes on a regular basis, could that have any impact upon indexing operations (e.g. indexWriter.close())? My understanding is that optimization only affects search performance, not indexing performance and to date we have not seen any need to optimize based upon the performance of the search queries. Thanks, Mark --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
