indexing performance problems

2009-06-08 Thread Mateusz Berezecki
s persistent in both 2.4.1 and 2.9-dev versions. Is there any known bug related to long running batch indexing processes that operate on large documents? In my case the single XML file is 20 GB and I'm just surprised how quickly the performance of the indexer degrades. Do you have any suggest

Re: indexing performance problems

2009-06-08 Thread Mateusz Berezecki
Hi Michael Thanks for a prompt response. On Mon, Jun 8, 2009 at 1:27 PM, Michael McCandless wrote: > This isn't normal. > > A mergeFactor of 150 is way too high; I'd put that back to 10 and see > if the problem persists.  Also make sure you're using > autoCommit=false, and try the suggestions her

Re: indexing performance problems

2009-06-08 Thread Mateusz Berezecki
Hi Michael, Thanks a lot for a hint. I'll test it out in a few hours and get back to you and/or the list. best, Mateusz On Mon, Jun 8, 2009 at 2:13 PM, Michael McCandless wrote: > On Mon, Jun 8, 2009 at 7:54 AM, Mateusz Berezecki wrote: > >> Thanks for a prompt response. &

Re: indexing performance problems

2009-06-10 Thread Mateusz Berezecki
more passes during the day. Mateusz On Wed, Jun 10, 2009 at 2:31 AM, Michael McCandless wrote: > Oh actually could you send this response to the full list (so they see > closure too)?  Thanks. > > Mike > > On Tue, Jun 9, 2009 at 6:32 PM, Mateusz Berezecki wrote: >> Hi Micha