Re: lucene 4.0 and DocumentsWriterPerThreadPool compared to lucene 3.4

2011-10-15 Thread Simon Willnauer
FYI - there is an issue related to merging likely causing this slowdown which are addressed in https://issues.apache.org/jira/browse/LUCENE-3515 for those who are interested in this issue. simon On Tue, Oct 11, 2011 at 4:01 PM, Marc Sturlese wrote: > Simon, > In this example I've set the DEFAULT

Re: lucene 4.0 and DocumentsWriterPerThreadPool compared to lucene 3.4

2011-10-11 Thread Marc Sturlese
Simon, In this example I've set the DEFAULT_MAX_THREAD_STATES of DocumentsWriterPerThreadPool to 1. I've debugged the code and I've made sure that ThreadAffinityDocumentsWriterThreadPool has the value set to 1 (as I was trying to make it behave similar to lucene 3.4 using a single thread). I'm ind

Re: lucene 4.0 and DocumentsWriterPerThreadPool compared to lucene 3.4

2011-10-11 Thread Simon Willnauer
marc, can you provide more info about your IndexWriterConfig you are using? maybe just call IndexWriterConfig#toString() and past it in? simon On Tue, Oct 11, 2011 at 2:14 PM, Marc Sturlese wrote: > I'm doing some performance test doing bulk indexing with lucene 4.0 and I'm > seeing weird resul

Re: lucene 4.0 and DocumentsWriterPerThreadPool compared to lucene 3.4

2011-10-11 Thread Mihai Caraman
Hey, you should compare with the ThreadedIndexWriter too :). I'll attach the source from Lucene in action SE manual and you can just replace the new IntexWriter(... with new ThreadedIndexWriter(... See if those results make a difference. Also I presume you don't have a single core cpu 2011/10/11

lucene 4.0 and DocumentsWriterPerThreadPool compared to lucene 3.4

2011-10-11 Thread Marc Sturlese
I'm doing some performance test doing bulk indexing with lucene 4.0 and I'm seeing weird results. I've read http://www.gossamer-threads.com/lists/lucene/java-dev/127190?do=post_view_threaded#127190 but I'm still having doubts. I'm building an index of 1G containing 1 milion docs. When building the