Using Lucene 3.5.0, on a 32-core machine, I have coded something shaped like:
make a writer on a RAMDirectory.
start:
Create a near-real-time searcher from it.
farm work out to multiple threads, each of which performs a search
and retrieves some docs.
When all are done, write some new docs.
back to start.
The returns of adding threads diminish faster than I would like.
According to YourKit, a major contribution when I try 16 is conflict
on the RAMFile monitor.
The conflict shows five Lucene Merge Threads holding the monitor, plus
my own threads. I'm not sure that I'm interpreting this correctly;
perhaps there were five different occasions when a merge thread
blocked my threads.
In any case, I'm fairly stumped as to how my threads manage to
materially block each other, since the synchronized methods used on
the search side in RAMFile are pretty tiny.
YourKit claims that the problem is in RAMFile.numBuffers, but I have
not been able to catch this being called in a search.
I did spot the following backtrace.
In any case, I'd be grateful if anyone could tell me if this is a
familiar story or one for which there's a solution.
RAMFile.getBuffer(int) line: 75
RAMInputStream.switchCurrentBuffer(boolean) line: 107
RAMInputStream.seek(long) line: 144
SegmentNorms.bytes() line: 163
SegmentNorms.bytes() line: 143
ReadOnlySegmentReader(SegmentReader).norms(String) line: 599
TermQuery$TermWeight.scorer(IndexReader, boolean, boolean) line: 107
BooleanQuery$BooleanWeight.scorer(IndexReader, boolean, boolean) line:
298
IndexSearcher.search(Weight, Filter, Collector) line: 577
IndexSearcher.search(Weight, Filter, int, Sort, boolean) line: 517
IndexSearcher.search(Weight, Filter, int, Sort) line: 487
IndexSearcher.search(Query, Filter, int, Sort) line: 400
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
