Re: Corrupted Indexes Under Lucene 2.3 (and 2.3.1)

2008-03-01 Thread Tyler V
Thanks for the reply Yonik. Our workflow is as follows: We build a very large document and put the document on a queue to be added to our "complete" index. This queue is serviced by a separate thread, which actually adds the document to the "complete" index. Once the document has been placed on

Re: Corrupted Indexes Under Lucene 2.3 (and 2.3.1)

2008-03-01 Thread Michael McCandless
Note that there are actually two concurrency issues to guard against here: * Document itself cannot be changed (fields added or removed) from multiple threads without external synchronization. * Document cannot be changed from one thread while another thread is calling writer.addDoc

Re: Corrupted Indexes Under Lucene 2.3 (and 2.3.1)

2008-02-29 Thread Yonik Seeley
On Fri, Feb 29, 2008 at 7:05 PM, Tyler V <[EMAIL PROTECTED]> wrote: > Mike -- Thanks so much for the prompt reply. > > You are right, we are accessing these documents with multiple threads > (and have always been). However, I am wondering if the increased > indexing speed in 2.3 has revealed a h

Re: Corrupted Indexes Under Lucene 2.3 (and 2.3.1)

2008-02-29 Thread Tyler V
Mike -- Thanks so much for the prompt reply. You are right, we are accessing these documents with multiple threads (and have always been). However, I am wondering if the increased indexing speed in 2.3 has revealed a hidden concurrency issue. I am going to add in some additional concurrency check

Re: Corrupted Indexes Under Lucene 2.3 (and 2.3.1)

2008-02-29 Thread Michael McCandless
Not good! (I'm sorry). That first exception is worrisome. It's the root cause here. Can you describe your documents? That exception, if I'm reading it right, seems to imply that you have documents with 4762 fields. Is that right? Are you using multiple threads? Is it possible that yo

Corrupted Indexes Under Lucene 2.3 (and 2.3.1)

2008-02-29 Thread Tyler V
After upgrading to Lucene 2.3 (and subsequently 2.3.1), our application has experienced sporadic index corruptions on our larger (and more frequently updated) indexes. These indexes experienced no corruptions under any prior version of Lucene (which we have been using for several years). The patte

RE: Corrupted indexes

2005-08-30 Thread Eric Bressler
, or do you get multiple items in the index? > > -Oorspronkelijk bericht- > Van: Eric Bressler [mailto:[EMAIL PROTECTED] > Verzonden: maandag 29 augustus 2005 23:18 > Aan: java-user@lucene.apache.org > Onderwerp: Corrupted indexes > > I am running lucene 1.4.3 and I

RE: Corrupted indexes

2005-08-30 Thread Peter Veentjer - Anchor Men
: Corrupted indexes I am running lucene 1.4.3 and I have a situation where after adding and removing some objects, my index becomes corrupt. I have been careful to make sure that all adds and removes happen in a single thread (although I know that isn't necessarly needed) and it still occurs.

Corrupted indexes

2005-08-29 Thread Eric Bressler
I am running lucene 1.4.3 and I have a situation where after adding and removing some objects, my index becomes corrupt. I have been careful to make sure that all adds and removes happen in a single thread (although I know that isn't necessarly needed) and it still occurs. I am not sure how to go