Hi,
We have been using lucene 2.3.2 for years well (yes, we should upgrade).
Recently we encountered data corruption error when commiting IndexWriter:
///
background merge hit exception: _14b:c61262 _1ag:c11225 _1gb:c9411 _1gv:c905
_1gw:c50 _1gx:c50 _1gy:c50 _1gz:c50 _1h0:c31 into _1h1 [optimize]
java.io.IOException: background merge hit exception: _14b:c61262 _1ag:c11225
_1gb:c9411 _1gv:c905 _1gw:c50 _1gx:c50 _1gy:c50 _1gz:c50 _1h0:c31 into _1h1
[optimize]
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1787)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1727)
at org.apache.lucene.index.IndexWriter.optimize(IndexWriter.java:1707)
///
Then we use CheckIndex tool to analyze and found one segment (out of 13) having
problem:
///
test: stored fields.......ERROR [field data are in wrong format:
java.util.zip.DataFormatException: unknown compression method]
org.apache.lucene.index.CorruptIndexException: field data are in wrong format:
java.util.zip.DataFormatException: unknown compression method
at
org.apache.lucene.index.FieldsReader.uncompress(FieldsReader.java:605)
at org.apache.lucene.index.FieldsReader.addField(FieldsReader.java:392)
at org.apache.lucene.index.FieldsReader.doc(FieldsReader.java:259)
at
org.apache.lucene.index.SegmentReader.document(SegmentReader.java:934)
at org.apache.lucene.index.IndexReader.document(IndexReader.java:844)
at
org.apache.lucene.index.CheckIndex.testStoredFields(CheckIndex.java:702)
at org.apache.lucene.index.CheckIndex.checkIndex(CheckIndex.java:517)
at org.apache.lucene.index.CheckIndex.main(CheckIndex.java:898)
///
Our stored fields are very simple (just id and short title, more fields are
only for search).
Our data size is about 400MB and 83K documents. We started indexing from an
empty folder. Also we have been using lucene 2.3.2 for years and this is the 1st
time to encounter this issue?
Indexer is running in a linux box, "uname -a" returns:
Linux <our box name> 2.6.32-342-ec2 #43-Ubuntu SMP Wed Jan 4 18:22:42 UTC 2012
x86_64 GNU/Linux
We really appreciate any guidance,
Lisheng
