OK so the problem definitely comes from the slow merging.
I slightly increased the number merge count and thread to avoid the problem
described previously. But as expected, it just delayed it !
results : 75 minutes to index the 33GB xml file, and 150 minutes to finish
the merge after indexer.close
Hello,
First many thanks for getting in touch.
Michael McCandless-2 wrote:
>
> It looks like merging is running too slowly in your environment,
> relative to indexing; all of your indexing threads are stuck wanting
> to launch a new merge but there's already the max allowed (1)
> concurrent mer
I tried to switch back to Lucene 3.2.0, same configuration, and I encountered
the same problem, be at a later stage.
See http://piratepad.net/ep/pad/export/ro.kMgHIoReJ2w/rev.2?format=txt here
the thread dump .
--
View this message in context:
http://lucene.472066.n3.nabble.com/Thread-locking-w
Hi All,
I'm currently testing the new DocumentsWriterPerThread in Lucene 4.0.0
(latest build). I use the wikipedia full english article dump as a source
for indexing and the ThreadedIndexWriter implementation proposed in LIA to
achieve concurrent indexing.
Indexing performance seems good at the be
