Re: Indexing becomes slow with time

2009-04-30 Thread mark harwood
ckson To: java-user@lucene.apache.org Sent: Thursday, 30 April, 2009 14:46:21 Subject: Re: Indexing becomes slow with time This is surprising behavior, which is another way of saying that, given what you've said so far, this shouldn't be happening. I'd really look at system metrics

Re: Indexing becomes slow with time

2009-04-30 Thread Erick Erickson
This is surprising behavior, which is another way of saying that, given what you've said so far, this shouldn't be happening. I'd really look at system metrics, like whether you're swapping etc. In particular you might want to try varying how big you allow your memory footprint to grow before you f

Re: Indexing becomes slow with time

2009-04-30 Thread liat oren
Yes, I do run optimize... I did start looking at these tips in the last few days, but didn't think the optimize makes it so slow. Thanks! 2009/4/30 Ian Lea > Are you maybe running optimize after every n documents? There are > lots of tips in > http://wiki.apache.org/lucene-java/ImproveIndexin

Re: Indexing becomes slow with time

2009-04-30 Thread Ian Lea
Are you maybe running optimize after every n documents? There are lots of tips in http://wiki.apache.org/lucene-java/ImproveIndexingSpeed. -- Ian. On Thu, Apr 30, 2009 at 8:29 AM, liat oren wrote: > Hi, > > I noticed that when I start to index, it indexes 7 documents a second. After > 30 minu

Indexing becomes slow with time

2009-04-30 Thread liat oren
Hi, I noticed that when I start to index, it indexes 7 documents a second. After 30 minutes it goes down to 3 documents a second. After two hours it becomes very slow (I stopped it when it arrived to 320MB and did 1 document in almost a minute) As you can see, it happens only after 2000, 3000 doc