Re: Lucene index integrity during a system crash

2005-07-16 Thread jian chen
Thanks Otis and Nikhil for your confirmation. I am more confident about the Lucene index integrity. Nikhil, I don't see th reason why there is a corrupted .fdx file. Could it be caused by multi-threaded access to the index? Otis, I don't remember I asked about locking questions the other day. I t

Re: Lucene and numerical fields search

2005-07-16 Thread Otis Gospodnetic
Hi Ray, If you can share your BitSetQuery and QP modifications, feel free to suggest them for inclusion to the core or contrib (by attaching them to a new entry in Bugzilla). Otis --- Ray Tsang <[EMAIL PROTECTED]> wrote: > It seems TooManyClauses is a potential problem for any query that > exp

Re: Lucene index integrity during a system crash

2005-07-16 Thread Otis Gospodnetic
Hi Jian, Yes, I think what you describes is correct. You may end up with some "junk index segments" in the index directory, but as long as they are not recorded in segments file, they are irrelevant. Otis P.S. Did you ask you locking in Lucene the other day? --- jian chen <[EMAIL PROTECTED]> w

Re: Lucene index integrity during a system crash

2005-07-16 Thread Nikhil Goel
hey Jian, Thats a very good thread to start and we faced the similar situation in our production system where Lunce Index got actually corrupted coz of non-atomiticity of wrting the index. Your observation is correct and the only problem that could happen is there will be zombie segments in your

Re: Lucene index integrity during a system crash

2005-07-16 Thread jian chen
Hi, Otis, Thanks for your email. As this is very important for using Lucene in our production system, I looked at the code to try to understand. Here is my observation why the index won't be corrupted during a system crash. In the IndexWriter.java mergeSegments(...) method, there are two lines: s

RE: Searching for similar documents

2005-07-16 Thread Otis Gospodnetic
They should work with the version in SVN, as well as with 1.4.3. Otis --- "Kadlabalu, Hareesh" <[EMAIL PROTECTED]> wrote: > Thanks Otis! > > Are there Lucene 1.4 compatiable version of these classes? > > Thanks very much > -Hareesh > > -Original Message- > From: Otis Gospodnetic > To

RE: Searching for similar documents

2005-07-16 Thread Kadlabalu, Hareesh
Thanks Otis! Are there Lucene 1.4 compatiable version of these classes? Thanks very much -Hareesh -Original Message- From: Otis Gospodnetic To: java-user@lucene.apache.org Sent: 7/16/2005 3:30 AM Subject: Re: Searching for similar documents We've got this in Lucene's contrib/: $ ll con

Re: Lucene index integrity during a system crash

2005-07-16 Thread Otis Gospodnetic
The only corruption that I've seen mentioned on this list so far was the corruption of the segments file, and even that people have been able to manually edit with a hex editor. Otis --- jian chen <[EMAIL PROTECTED]> wrote: > Hi, > > I know Lucene does not have transaction support at this stag

Re: Searching for similar documents

2005-07-16 Thread Otis Gospodnetic
We've got this in Lucene's contrib/: $ ll contrib/similarity/src/java/org/apache/lucene/search/similar/*java -rwxrwxr-x 1 otis otis 30431 Jul 9 09:20 MoreLikeThis.java* -rwxrwxr-x 1 otis otis 3612 Mar 16 17:31 SimilarityQueries.java* Otis --- "Kadlabalu, Hareesh" <[EMAIL PROTECTED]> wrote: