segment exists in external directory yet the MergeScheduler executed the merge in a separate thread

[Anthony Urso]

I have implemented a MapReduce job to merge a bunch of Lucene 2.3.2
indices together, but the reducers randomly fail with the following
unchecked exception after thousands of successful merges:

org.apache.lucene.index.MergePolicy$MergeException: segment "_0 exists
in external directory yet the MergeScheduler executed the merge in a
separate thread
        at 
org.apache.lucene.index.IndexWriter.copyExternalSegments(IndexWriter.java:2362)
        at 
org.apache.lucene.index.IndexWriter.addIndexesNoOptimize(IndexWriter.java:2307)

Anyone know what would cause such a thing?

Here is the relevant code:

  IndexWriter writer = new IndexWriter(FSDirectory.getDirectory(name),
new StandardAnalyzer());

  Directory[] dir = new Directory[1];

  for (String p: paths) {
    dir[0] = FSDirectory.getDirectory(p);

    writer.addIndexesNoOptimize(dir);
  }

  writer.optimize();

  writer.close();

Cheers,
Anthony

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]