Is that a local file system, or a network share? On Thu, Aug 13, 2009 at 1:07 PM, rishisinghal <singhal.ri...@gmail.com>wrote:
> > >>Is there any chance that two writers are open on this directory? > No, thats not true. > > >>something external to Lucene is removing files from the directory. > No this also has rare chances as I am the owner of these files and other > then me no one can delete the, :-) > > Here are all the files in the index directory: > segments.gen;1 segments_1.;1 segments_2.;1 _0.fdt;1 > _0.fdx;1 _37.cfs;1 _4c.cfs;1 _61.cfs;1 > _9e.cfs;1 _ay.cfs;1 _az.cfs;1 _b0.cfs;1 > _b2.tis;1 _b3.fdt;1 _b4.fdt;1 > > I see _37 present. > > Regards, > Rishi > > > Michael McCandless-2 wrote: > > > > Is there any chance that two writers are open on this directory? Or, > > something external to Lucene is removing files from the directory. > > > > It looks like there were at least two missing files (_37 > > > > On Thu, Aug 13, 2009 at 5:19 AM, rishisinghal<singhal.ri...@gmail.com> > > wrote: > >> > >> Hi, > >> > >> I am trying to index documents and when all is complete and optimize is > >> called I get > >> > >> IFD [main]: setInfoStream > >> > deletionpolicy=org.apache.lucene.index.keeponlylastcommitdeletionpol...@4fced0 > >> IW 0 [main]: setInfoStream: > >> dir=org.apache.lucene.store.FSDirectory@/SYS$SYSDEVICE/RISHI/melon_1600 > >> autoCommit=false mergePolicy=org > >> .apache.lucene.index.logbytesizemergepol...@92cb49 > >> mergescheduler=org.apache.lucene.index.concurrentmergeschedu...@6038e4 > >> ramBufferS > >> izeMB=16.0 maxBufferedDocs=5 maxBuffereDeleteTerms=-1 > >> maxFieldLength=10000 > >> index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0 > >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > >> IW 0 [main]: optimize: index now _37:c465->_0 _4c:c165->_0 _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > >> IW 0 [main]: flush: segment=null docStoreSegment=null docStoreOffset=0 > >> flushDocs=false flushDeletes=true flushDocStores=false numD > >> ocs=0 numBufDelTerms=0 > >> IW 0 [main]: index before flush _37:c465->_0 _4c:c165->_0 _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > >> IW 0 [main]: add merge to pendingMerges: _61:c245->_0 _9e:c485->_0 > >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 [optimize] [total 1 pending] > >> IW 0 [main]: CMS: now merge > >> IW 0 [main]: CMS: index: _37:c465->_0 _4c:c165->_0 _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > >> IW 0 [main]: CMS: consider merge _61:c245->_0 _9e:c485->_0 > _ay:c225->_0 > >> _az:c5->_0 _b0:c5->_0 into _b5 [optimize] > >> IW 0 [main]: CMS: launch new thread [Lucene Merge Thread #73] > >> IW 0 [main]: CMS: no more merges pending; now return > >> IW 0 [Lucene Merge Thread #73]: CMS: merge thread: start > >> IW 0 [Lucene Merge Thread #73]: now merge > >> merge=_61:c245->_0 _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into > >> _b5 [optimize] > >> merge=org.apache.lucene.index.mergepolicy$oneme...@3725db > >> index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 _9e:c485->_0 _ay:c225->_0 > >> _az:c5->_0 _b0:c5->_0 > >> IW 0 [Lucene Merge Thread #73]: merging _61:c245->_0 _9e:c485->_0 > >> _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize] > >> IW 0 [Lucene Merge Thread #73]: hit exception during merge > >> Exception in thread "Lucene Merge Thread #73" > >> org.apache.lucene.index.MergePolicy$MergeException: > >> java.io.FileNotFoundException: /SY > >> S$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) > >> at > >> > org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:309) > >> at > >> > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:286) > >> Caused by: java.io.FileNotFoundException: > >> /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) > >> at java.io.RandomAccessFile.open(Native Method) > >> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) > >> at > >> > org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) > >> at > >> > org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) > >> at > >> org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488) > >> at > >> > org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70) > >> at > >> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321) > >> at > >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306) > >> at > >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260) > >> at > >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220) > >> at > >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884) > >> at > >> > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205) > >> at > >> > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260) > >> Aug 13, 2009 2:40:56 PM org.hp.openvms.lucene.index.FileIndexerJSP > >> jspMain > >> SEVERE: Indexing not completebackground merge hit exception: > _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [ > >> optimize] > >> java.io.IOException: background merge hit exception: _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 into _b5 [optimize > >> ] > >> at > >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2273) > >> at > >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2218) > >> at > >> org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:2198) > >> at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown > >> Source) > >> at org.hp.openvms.lucene.index.FileIndexer.main(Unknown Source) > >> Caused by: java.io.FileNotFoundException: > >> /SYS$SYSDEVICE/RISHI/melon_1600/_61.cfs (i/o error (errno:5)) > >> at java.io.RandomAccessFile.open(Native Method) > >> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) > >> at > >> > org.apache.lucene.store.FSDirectory$FSIndexInput$Descriptor.<init>(FSDirectory.java:552) > >> at > >> > org.apache.lucene.store.FSDirectory$FSIndexInput.<init>(FSDirectory.java:582) > >> at > >> org.apache.lucene.store.FSDirectory.openInput(FSDirectory.java:488) > >> at > >> > org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70) > >> at > >> org.apache.lucene.index.SegmentReader.initialize(SegmentReader.java:321) > >> at > >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:306) > >> at > >> org.apache.lucene.index.SegmentReader.get(SegmentReader.java:260) > >> at > >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4220) > >> at > >> org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:3884) > >> at > >> > org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:205) > >> at > >> > org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:260) > >> IW 0 [main]: now flush at close > >> IW 0 [main]: flush: segment=null docStoreSegment=null docStoreOffset=0 > >> flushDocs=false flushDeletes=true flushDocStores=false numD > >> ocs=0 numBufDelTerms=0 > >> IW 0 [main]: index before flush _37:c465->_0 _4c:c165->_0 _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > >> IW 0 [main]: CMS: now merge > >> IW 0 [main]: CMS: index: _37:c465->_0 _4c:c165->_0 _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 > >> IW 0 [main]: CMS: no more merges pending; now return > >> IW 0 [main]: now call final commit() > >> IW 0 [main]: startCommit(): start sizeInBytes=0 > >> IW 0 [main]: startCommit index=_37:c465->_0 _4c:c165->_0 _61:c245->_0 > >> _9e:c485->_0 _ay:c225->_0 _az:c5->_0 _b0:c5->_0 changeCount=11 > >> 95 > >> IW 0 [main]: now sync _37.cfs > >> IW 0 [main]: hit exception while closing > >> Exception in thread "main" java.io.FileNotFoundException: > >> /SYS$SYSDEVICE/RISHI/melon_1600/_37.cfs (i/o error (errno:5)) > >> at java.io.RandomAccessFile.open(Native Method) > >> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:231) > >> at org.apache.lucene.store.FSDirectory.sync(FSDirectory.java:456) > >> at > >> org.apache.lucene.index.IndexWriter.startCommit(IndexWriter.java:4590) > >> at > >> org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:3369) > >> at > >> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:1654) > >> at > >> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1602) > >> at > >> org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1578) > >> at org.hp.openvms.lucene.index.FileIndexerJSP.jspMain(Unknown > >> Source) > >> at org.hp.openvms.lucene.index.FileIndexer.main(Unknown Source) > >> > >> > >> > >> I have setMaxBufferedDocs = 5 and setMergeFactor=5. I was trying > >> different > >> combinations of these to see if that helps but was unsuccessful. > >> > >> > >> Regards, > >> Rishi > >> -- > >> View this message in context: > >> > http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24951667.html > >> Sent from the Lucene - Java Users mailing list archive at Nabble.com. > >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: java-user-h...@lucene.apache.org > >> > >> > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > > For additional commands, e-mail: java-user-h...@lucene.apache.org > > > > > > > > -- > View this message in context: > http://www.nabble.com/Indexer-crashes-with-%22hit-exception-during-merge%22-tp24951667p24952295.html > Sent from the Lucene - Java Users mailing list archive at Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
