Re: best practice on too many files vs IO overhead

g.  I would triple check > that all readers are being closed. > > Or... if you list the index directory, how many files do you see? > > Mike > > On Fri, Nov 27, 2009 at 5:48 AM, Istvan Soos wrote: >> On Fri, Nov 27, 2009 at 11:37 AM, Michael McCandless >> wrote: >&g

Re: best practice on too many files vs IO overhead

On Fri, Nov 27, 2009 at 11:37 AM, Michael McCandless wrote: > Are you sure you're closing all readers that you're opening? Absolutely. :) (okay, never say this, but I had bugz because of this previously so I'm pretty sure that one is ok). > It's surprising with normal usage of Lucene that you'd

best practice on too many files vs IO overhead

Hi, I've a requirement that involves frequent, batched update of my Lucene index. This is done by a memory queue and process that periodically wakes and process that queue into the Lucene index. If I do not optimize my index, I'll receive "too many open files" exception (yeah, right, I can get th

best way to ensure IndexWriter won't corrupt the index?

Hi, What are the typical scenarios when the index will go corrupt? E.g. can a simple JVM crash during indexing will cause it? What are the best way to minimalize the possibility of corrupt index? Copy the directory before indexing / then flipping the pointers? I'm using Lucene 2.9. Thanks, I