Re: "read past EOF" when merge

2012-11-07 Thread superruiye
Sorry,I can't reproduce in local... "If at any time you accidentally had two writers open on the same index, it could have created this corruption. " I use singleton to create/get IndexWriter ,and change LockFactory to SimpleFSLockFactory (NativeFSLockFactory before) .Maybe a long time it was run

RE: "read past EOF" when merge

2012-11-05 Thread Markus Jelsma
https://issues.apache.org/jira/browse/SOLR-4032 -Original message- > From:Mark Miller > Sent: Sat 03-Nov-2012 14:20 > To: java-user@lucene.apache.org > Subject: Re: "read past EOF" when merge > > Can you file a JIRA Markus? This is probably related

Re: "read past EOF" when merge

2012-11-03 Thread Mark Miller
a:932) > > > Markus > > -Original message----- >> From:Michael McCandless >> Sent: Fri 02-Nov-2012 11:46 >> To: java-user@lucene.apache.org >> Subject: Re: "read past EOF" when merge >> >> Are you able to reproduce the corruption

RE: "read past EOF" when merge

2012-11-02 Thread Markus Jelsma
No this is not using NFS but EXT3 on SSD. Thanks -Original message- > From:Michael McCandless > Sent: Fri 02-Nov-2012 16:22 > To: java-user@lucene.apache.org > Subject: Re: "read past EOF" when merge > > On Fri, Nov 2, 2012 at 6:53 AM, Markus Jelsma &

Re: "read past EOF" when merge

2012-11-02 Thread Michael McCandless
On Fri, Nov 2, 2012 at 6:53 AM, Markus Jelsma wrote: > Hi, > > For what it's worth, we have seen similar issues with Lucene/Solr from this > week's trunk. The issue manifests itself when it want to replicate. The > servers have not been taken offline and did not crash when this happenend. > > 20

RE: "read past EOF" when merge

2012-11-02 Thread Markus Jelsma
nHandler$3.write(ReplicationHandler.java:932) Markus -Original message- > From:Michael McCandless > Sent: Fri 02-Nov-2012 11:46 > To: java-user@lucene.apache.org > Subject: Re: "read past EOF" when merge > > Are you able to reproduce the corruption? &

Re: "read past EOF" when merge

2012-11-02 Thread Michael McCandless
Are you able to reproduce the corruption? If at any time you accidentally had two writers open on the same index, it could have created this corruption. Writing to an index over NFS ought to be OK, however, it's not well tested. You should use SimpleFSLockFactory (not the default NativeFSLockFac

Re: "read past EOF" when merge

2012-11-01 Thread superruiye
oh ,thx,I don't know CheckIndex before...and I use to fix my error index,it is OK... I use NFS to share my index,and no change to the LogFactory. How could I avoid this problem,and not only fix after it was broken suddenly? -- View this message in context: http://lucene.472066.n3.nabble.com/rea

Re: "read past EOF" when merge

2012-11-01 Thread Michael McCandless
Well somehow you have a corrupt index. I'd be very interested in how that happened :) You need to run CheckIndex with -fix (it's a command-line tool, too) to remove that bad segment else some search will eventually hit an exception on that segment ... and merging can never be done with that segme

Re: "read past EOF" when merge

2012-10-31 Thread pliu367
Hi Mike, I got the same problem as above. My search project is running on two servers(1 master for index writing and reading and 1 slaver for index reading), and the master server writes the index into a seperate disk(not belong to master) which is shared by the slaver. I have never changed the Loc

Re: "read past EOF" when merge

2012-10-31 Thread Michael McCandless
Run CheckIndex on the index? What filesystem is IndexWriter using to write to the index...? Have you changed the LockFactory on the Directory? Mike McCandless http://blog.mikemccandless.com On Tue, Oct 30, 2012 at 11:27 PM, superruiye wrote: > The exception "read past EOF" Bothering me a lon

Re: read past EOF

2010-08-11 Thread Michael McCandless
ards > Ganesh > > - Original Message - > From: "Michael McCandless" > To: > Sent: Wednesday, August 11, 2010 3:28 PM > Subject: Re: read past EOF > > > It looks like it may be this issue: > >    https://issues.apache.org/jira/browse/LUCENE-2593 > > Bu

Re: read past EOF

2010-08-11 Thread Ganesh
I am using Lucene 2.9.1 and there was no exception in the past. Regards Ganesh - Original Message - From: "Michael McCandless" To: Sent: Wednesday, August 11, 2010 3:28 PM Subject: Re: read past EOF It looks like it may be this issue: https://issues.apache.org/j

Re: read past EOF

2010-08-11 Thread Michael McCandless
It looks like it may be this issue: https://issues.apache.org/jira/browse/LUCENE-2593 But can you describe the circumstances leading up to this? EG were there any exceptions (eg disk full) before this one? Which version of Lucene? Mike On Wed, Aug 11, 2010 at 12:30 AM, Ganesh wrote: > He

Re: Read past EOF

2009-04-29 Thread Michael McCandless
I've opened https://issues.apache.org/jira/browse/LUCENE-1623 for this. Mike On Tue, Apr 28, 2009 at 10:15 AM, Michael McCandless wrote: > Ugh, indeed FieldInfos fails to properly read 2.3.x indices if the > field name contains non-ascii characters.  I'll open an issue, make a > test case and wo

Re: Read past EOF

2009-04-28 Thread Michael McCandless
Ugh, indeed FieldInfos fails to properly read 2.3.x indices if the field name contains non-ascii characters. I'll open an issue, make a test case and work out a fix. Hmm. Thanks for raising this! Mike On Tue, Apr 28, 2009 at 7:53 AM, Mike Streeton wrote: > I have an index that works fine on L

RE: Read past EOF

2009-04-28 Thread Mike Streeton
An update, I have managed to get it to not fail by debugging and changing the value of org.apache.lucene.store.InputIndex.preUTF8Strings = true. The value is always false when it fails. Mike -Original Message- From: Mike Streeton [mailto:mike.stree...@connexica.com] Sent: 28 April 200

Re: read past EOF

2006-09-08 Thread Michael McCandless
Bhavin Pandya wrote: It sounds like you're working with the index correctly, so I don't have any other ideas on why you're getting CFS files that are truncated. I would wory about the "cp" step filling up disk, but if you're nowhere near filling up disk that's not the root cause here. I h

Re: read past EOF

2006-09-08 Thread Bhavin Pandya
like "no space left"...but when i gone through all the log i tracked it sucessfully. Thanks for your help - Bhavin pandya - Original Message - From: "Michael McCandless" <[EMAIL PROTECTED]> To: Sent: Friday, September 01, 2006 5:07 PM Subject: R

Re: read past EOF

2006-09-01 Thread Michael McCandless
Yes I am sure only one writer at a time accessing index. no i am not getting any other exception. and there is no problem of disk space also. right now i have backcopy of indexes so whenever one index got corrupted i m replacing with backup one and starting the indexer again from that durat

Re: read past EOF

2006-08-30 Thread Bhavin Pandya
age - From: "Michael McCandless" <[EMAIL PROTECTED]> To: Sent: Thursday, August 31, 2006 8:01 AM Subject: Re: read past EOF Bhavin Pandya wrote: My guess is ..."One of my index is got corrupted so whenever I am trying to search the index or optimize the index or mer

Re: read past EOF

2006-08-30 Thread Michael McCandless
Bhavin Pandya wrote: My guess is ..."One of my index is got corrupted so whenever I am trying to search the index or optimize the index or merging the multiple index ...It will throws same exception but from different class...sometime from IndexReader or sometime from IndexWriter depends on how

Re: read past EOF

2006-08-30 Thread Bhavin Pandya
index. Its not multi-threaded application...so I am sure only one thread at a time accessing the index... Thanks. Bhavin - Original Message - From: "Michael McCandless" <[EMAIL PROTECTED]> To: Sent: Wednesday, August 30, 2006 4:07 PM Subject: Re: read past EOF Bhavin Pandy

Re: read past EOF

2006-08-30 Thread Michael McCandless
Bhavin Pandya wrote: I am running lucene 1.9 on unix machine...updating my index very frequentlyafter few updation it says "read past eof" I know this exception generally comes when one of the index got corrupted...but i dont know why it got corrupted ? may be mine code problem but i am

Re: Read past EOF error in Windows

2006-03-23 Thread Raghavendra Prabhu
Check Whether it has got anything to do with UTF There is a new line difference between windows and linux Rgds Prabhu On 3/24/06, Chris Cain <[EMAIL PROTECTED]> wrote: > > No that doesnt seem to be the problem. > > Anyone have any other ideas? > > On Tue, 21 Mar 2006 [EMAIL PROTECTED] > > I had

Re: Read past EOF error in Windows

2006-03-23 Thread Chris Cain
No that doesnt seem to be the problem. Anyone have any other ideas? On Tue, 21 Mar 2006 [EMAIL PROTECTED] I had a problem in the past with security on the folder where your index is located...but your error does not seem to show that ... I would check anyway though... -Original Message-

Re: Read past EOF error in Windows

2006-03-21 Thread msftblows
I had a problem in the past with security on the folder where your index is located...but your error does not seem to show that ... I would check anyway though... -Original Message- From: Chris Cain <[EMAIL PROTECTED]> To: java-user@lucene.apache.org Sent: Tue, 21 Mar 2006 15:33:26 +00

Re: read past EOF

2005-08-27 Thread jian chen
Hi, It seems this problem only happens when the index files get really large. Could it be because java has trouble handling very large files on windows machine (guess there is max file size on windows)? In Lucene, I think there is a maxDoc kind of parameter that you can use to specify, when th