Yesterday, when typing a reply I noticed an error ... the CheckIndex.check
was not checking a good directory.. so the segments not found error was not
correct. I apologize. I made the correction and this is the correct error
that occurs when an i/o error is thrown. I ran the CheckIndex.check method
after the i/o exception was caught:
Segments file=segments_296 numSegments=1 version=FORMAT_SHARED_DOC_STORE
[Lucene 2.3]
1 of 1: name=+1dq docCount=1788454
compund=true
numFiles=1
size (MB) = 331.125
no deletions
test: open reader....... OK
test: fields, norms...... OK [4 fields]
test: terms, freq, prox...2008-11-05 15:12:33,357 INFO Cache:195 -
Purging 0 of 4 Cached results.
2008-11-05 15:12:33,358 INFO Cache: 196 - 956018816 of 1073676288 bytes of
memory are currently free
OK [3409802 terms; 19311204 terms/docs pairs; 19842525 tokens]
test: stored fields....... Wed Nov 05 15:12:59 UTC 2008: Purging Query
Cache 0 of 9 query cached results.
Wed Nov 05 15:12:59 UTC 2008: 955521656 of 1073676288 bytes of memory are
currently free.
FAILED
WARNING: would remove reference to this segment (-fix was not
specified); full exception:
java.io.IOException: Input/Output error
at java.io.RandomAccessfile.readBytes(Native Method)
at java.io.RandomAccessFile.read(RandomAccessfile.java:315)
at
org.apache.lucene.store.FSDirectory$FSIndexInput.readInternal(FSDirectory.java:550)
at
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:131)
at
org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:240)
at
org.apache.lucene.store.BufferedIndexInput.refill(bufferedIndexInput.java:152)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:38)
at org.apache.lucene.store.IndexInput.readChars(IndexInput.java:126)
at org.apache.lucene.store.IndexInput.readString(IndexInput.java:112)
at org.apache.lucene.index.FieldsReader.addfield(FieldsReader.java:324)
at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:166)
at org.apache.lucene.index.SegmentReader.document(SegmentReader.java:659)
at org.apache.lucene.index.IndexReader.document(IndexReader.java:525)
at org.apache.lucene.index.CheckIndex.check(checkIndex.java:234)
at
gminer.rest.content.RestContentSearchServlet.doGet(RestContentSearchServlet.java:269)
.................. continues on and then
WARNING: 1 broken segments detected
WARNING: 1788454 documents would be lost if -fix were specified
NOTE: would write new segments file [-fix was not specified]
************************************************
Then I wrote out the return status from the check method call and it was "
false".
The Normal Input/Output error follows...
NOW, I am leaving the IndexSearcher open for the current day and if I
resubmit the query... the query will most likely work the 2nd time... Is
there something to be done to the index so that there are not the errors
above?
I appreciate all your help.
Julie
JulieSoko wrote:
>
> I am narrowing down this problem that I have had for a week now... I am
> using lucene version 2.3.1 and 64 bit java versio 1.5.0-12-b04 running on
> Linux box. We are merging indexes every 60 seconds and there are 1..*
> searches occuring at anytime on the indexes. The problem is that we will
> get an Input /Output error trying to read the index for a search
> randomly... say every 5th search. I have posted the error before, but
> have narrowed it down, I believe, to a merge issue.
>
> This is the error that a searcher will output at random times:
> java.io.IOException: Input/output error
> java.io.RandomAccessFile.readBytes(Native Method)
> java.io.RandomAccessFile.read(RandomAccessFile.java:315)
> at
> org.apache.lucene.store.FSDirecotry$FSIndexInput.readInternal(FSDirectory.java:550)
> at
> org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedInputInput.java:131)
> at
> org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:240)
> at
> org.apache.lucene.instoreBufferedIndexInput.refill(BufferedIndexInput.java:
> 152)
> at
> org.apache.lucene.instoreBufferedIndexInput.readByte(BufferedIndexInput.java:
> 152)
> at org.lucene.store.IndexInput.readVInt(IndexInput.java:76)
> at org.apache.lucene.index.TermBuffer.read(TermBuffer.java:63)
> at org.apache.lucene.index.SegmentTermEnum.next(SegmentTermEnum.java:123)
> at
> org.apache.lucene.index.SegmentTermEnum.scanTo(SegmentTermEnum.java:154)
> at
> org.apache.lucene.index.TermInfosReader.scanEnum(TermInfosReader.java:223)
> at org.apache.lucene.index.TermInfosReader.get(TermInfosReader.java:217)
> at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:678)
> at org.apache.lucene.search.IndexSearcher.docFreq(IndexSearcher.java:87)
> at org.apache.lucene.search.Searcher.docFreqs(searcher.java:118)
> at
> org.apache.lucene.search.MultiSearcher.createWeight(MultiSearcher.java:311)
> at org.apache.lucene.search.Searcher.search(Searcher.java:178)
>
>
> ******************************************************************************
> NOW, when I get the above Exception, I check the index using the
> CheckIndex.check method... As part of the check , this exception is
> thrown:
>
>
> Error: could not read any segments file in directory
> java.io.FileNotFoundException: no segments* file found in
> [EMAIL PROTECTED]/rt10/jetty/20081103
> at
> org.apache.lucene.index.SegmentInfos$findSegmentsFile.run(SegementInfos.java:587)
> .....
>
> Is there any point in the merging of indexes that the segment files are
> removed? If I rerun the search, right after this error occurs, the
> search is ok... I do open a new IndexSearcher...
>
> The IndexWriter code is this:
> IndexWriter combinedWriter = new IndexWriter(currentMergeDir, new
> StandardAnalyzer());
> combinedWriter.addIndexes(dirToMerge);
> combinedWriter.flush();
> combinedWriter.close();
>
>
> As you can see above, each time there is a merge a new IndexWriter is
> created, indexes added, flushed and closed.
> I know you are not supposed to have synchronization issues between writing
> and flushing , but could there be an issue when you are creating a new
> searcher at the instant where the files are merged and there are no
> segments in a dir???
>
> Thanks,
> Julie
>
>
>
>
>
>
>
>
>
--
View this message in context:
http://www.nabble.com/No-segment-files-found--Searcher-error-tp20305354p20343211.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
