Re: Incremental updates / slow searches.

2006-10-11 Thread Rickard Bäckman
Thanks for the suggestions. We tried to reduce the amount of times we open a new searcher with some progress. However a lot of our searches still times out. We are currently opening a new searcher and warms it up before doing the switch. We even map the fields we are using for deleting to the doc

Re: Incremental updates / slow searches.

2006-10-09 Thread Mathias Lux
Rickard Bäckman wrote: > Hi, > > we are using a search system based on Lucene and have recently tried to add > incremental updating of the index instead of building a new index every now > and then. However we now run into problems as our searches starts to take > very long time to complete. > >

Re: Incremental updates / slow searches.

2006-10-09 Thread Yonik Seeley
On 10/9/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: don't forget to optimize your index every now and then as well... deleting a document just marks it as "deleted" it still gets inspectected by every query during scoring at least once to see that it can skip it, optimizing is the only thing t

Re: Incremental updates / slow searches.

2006-10-09 Thread Chris Hostetter
quot; documents. : Date: Mon, 9 Oct 2006 13:49:34 -0400 : From: Yonik Seeley <[EMAIL PROTECTED]> : Reply-To: java-user@lucene.apache.org : To: java-user@lucene.apache.org : Subject: Re: Incremental updates / slow searches. : : The biggest thing would be to limit how often you open a new : Index

Re: Incremental updates / slow searches.

2006-10-09 Thread Yonik Seeley
The biggest thing would be to limit how often you open a new IndexSearcher, and when you do, warm up the new searcher in the background while you continue serving searches with the existing searcher. This is the strategy that Solr uses. There is also the issue of if you are analyzing/merging doc

Incremental updates / slow searches.

2006-10-09 Thread Rickard Bäckman
Hi, we are using a search system based on Lucene and have recently tried to add incremental updating of the index instead of building a new index every now and then. However we now run into problems as our searches starts to take very long time to complete. Our index is about 8-9GB large and we

Re: incremental updates

2006-05-23 Thread karl wettin
On Mon, 2006-05-22 at 13:07 -0700, Van Nguyen wrote: > I'm pretty new to lucene and was wondering if there are any resources on > how to do incremental updates in lucene. What do you mean by incremental updates? You add data to your corpus by using the

incremental updates

2006-05-22 Thread Van Nguyen
I'm pretty new to lucene and was wondering if there are any resources on how to do incremental updates in lucene. Thanks! Van Nguyen Wynne Systems, Inc. 19800 MacArthur Blvd., Suite 900 Irvine, CA 92612-2421 949.224.6300 ext 223 949.225.6540 (fax) 866.901.9284 (toll-free) www.wynnesystem