Do you have 2 separate segments files in that 2nd index? Which exact Lucene version is this?
Mike McCandless http://blog.mikemccandless.com On Thu, Sep 22, 2016 at 7:44 AM, Ziming Dong <dzm1016397...@gmail.com> wrote: > I used checkIndex to recover second index though I lost many docs in index, > but first index can't be read by checkIndex, error is > >> java -cp lucene-core-6.1.0.jar -ea:org.apache.lucene... >> org.apache.lucene.index.CheckIndex /Volumes/HPT8_56T/infomall-index/index0 >> Opening index @ /Volumes/HPT8_56T/infomall-index/index0 >> ERROR: could not read any segments file in directory >> org.apache.lucene.index.IndexFormatTooOldException: Format version is not >> supported (resource >> BufferedChecksumIndexInput(MMapIndexInput(path="/Volumes/HPT8_56T/infomall-index/index0/segments_5t3"))): >> 0 (needs to be between 1071082519 and 1071082519). This version of Lucene >> only supports indexes created with release 5.0 and later. >> at >> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:295) >> at >> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:284) >> at >> org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:507) >> at >> org.apache.lucene.index.CheckIndex.doCheck(CheckIndex.java:2595) >> at org.apache.lucene.index.CheckIndex.doMain(CheckIndex.java:2497) >> at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:2423) > > > I use NFS, but I set mount option as mount -t nfs -o tcp,sync,retrans=10 > The index program has run 1 month without any problem before power failure. > > On Thu, Sep 22, 2016 at 6:06 PM, Michael McCandless > <luc...@mikemccandless.com> wrote: >> >> Hmm I'm no longer so sure this is an IW bug: on commit we fsync the >> pending_segments_N and then do an atomic rename to segments_N. >> >> Can you describe your IO system? Is it possible it does not implement >> fsync or atomic renames correctly? >> >> Also, your 2nd exception indices the segments_N file was intact but >> the .cfs file was corrupt, which is also hard to explain unless fsync >> isn't working on your IO system. >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> On Thu, Sep 22, 2016 at 5:10 AM, Michael McCandless >> <luc...@mikemccandless.com> wrote: >> > Sorry for the slow reply here. Curious that both of these exceptions >> > are from IW.init. I think this may be a real bug, caused by this: >> > >> > https://github.com/apache/lucene-solr/commit/981bfba841144d08df1d1a183d39fcd6f195ad56 >> > >> > I'll see if I can make a standalone test case showing this. >> > >> > If you open those indices with an IndexReader instead, does it succeed? >> > >> > If you run CheckIndex, what does it report? >> > >> > Mike McCandless >> > >> > http://blog.mikemccandless.com >> > >> > On Wed, Sep 14, 2016 at 1:22 AM, Ziming Dong <dzm1016397...@gmail.com> >> > wrote: >> >> I have 6 machine and 6 index directories, each machine builds index >> >> into >> >> one index directory. After power failure last night, two of those >> >> machine >> >> can't start index program. >> >> >> >> one error is >> >> >> >>> INFO: 2016-09-14 12:31:38 [main] >> >>> sewm.bdbox.search.InfomallIndexer$Builder:ignoreCollectionsFile(227): >> >>> Loaded 2146 ignored collections from >> >>> /mnt/HPT8_56T/infomall-index/index0/ignored_collections.txt >> >>> ERROR: 2016-09-14 12:31:39 [main] sewm.bdbox.util.LogUtil:error(71): >> >>> org.apache.lucene.index.IndexFormatTooOldException: Format version is >> >>> not >> >>> supported (resource >> >>> >> >>> BufferedChecksumIndexInput(MMapIndexInput(path="/mnt/HPT8_56T/infomall-index/index0/segments_5t3"))): >> >>> 0 (needs to be between 1071082519 and 1071082519). This version of >> >>> Lucene >> >>> only supports indexes created with release 5.0 and later. >> >>> at >> >>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:295) >> >>> at >> >>> org.apache.lucene.index.SegmentInfos.readCommit(SegmentInfos.java:284) >> >>> at >> >>> org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:910) >> >>> at >> >>> sewm.bdbox.search.InfomallIndexer.<init>(InfomallIndexer.java:60) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:28) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:21) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer$Builder.build(ThreadedInfomallIndexer.java:72) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer.main(ThreadedInfomallIndexer.java:129) >> >> >> >> >> >> another is >> >> >> >> INFO: 2016-09-14 01:11:06 [main] >> >>> sewm.bdbox.search.InfomallIndexer$Builder:ignoreCollectionsFile(227): >> >>> Loaded 8575 ignored collections from >> >>> /mnt/HPT8/infomall-index/index5/ignored_collections.txt >> >>> ERROR: 2016-09-14 01:11:09 [main] sewm.bdbox.util.LogUtil:error(71): >> >>> org.apache.lucene.index.CorruptIndexException: codec footer mismatch >> >>> (file >> >>> truncated?): actual footer=0 vs expected footer=-1071082520 >> >>> >> >>> (resource=MMapIndexInput(path="/mnt/HPT8/infomall-index/index5/_1kqn.cfs")) >> >>> at >> >>> org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:448) >> >>> at >> >>> >> >>> org.apache.lucene.codecs.CodecUtil.retrieveChecksum(CodecUtil.java:433) >> >>> at >> >>> >> >>> org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.<init>(Lucene50CompoundReader.java:86) >> >>> at >> >>> >> >>> org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:71) >> >>> at >> >>> >> >>> org.apache.lucene.index.IndexWriter.readFieldInfos(IndexWriter.java:1016) >> >>> at >> >>> >> >>> org.apache.lucene.index.IndexWriter.getFieldNumberMap(IndexWriter.java:1033) >> >>> at >> >>> org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:938) >> >>> at >> >>> sewm.bdbox.search.InfomallIndexer.<init>(InfomallIndexer.java:60) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:28) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer.<init>(ThreadedInfomallIndexer.java:21) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer$Builder.build(ThreadedInfomallIndexer.java:72) >> >>> at >> >>> >> >>> sewm.bdbox.search.ThreadedInfomallIndexer.main(ThreadedInfomallIndexer.java:129) >> >>> >> >> >> >> >> >> it seems 1071082519 is a special number. >> >> >> >> - - >> >> >> >> Ziming Dong >> >> *http://suiyuan2009.github.io/ <http://suiyuan2009.github.io/>* > > > > > -- > > Ziming Dong > http://suiyuan2009.github.io/ > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
