Re: Searching on a Rapidly changing Index

2007-05-26 Thread Yonik Seeley
On 5/24/07, Joe Shaw <[EMAIL PROTECTED]> wrote: On 5/24/07, Erick Erickson <[EMAIL PROTECTED]> wrote: > If that's unacceptable, you can *still* open up a new reader in the > background and warm it up before using it. "immediately" then > becomes 5-10 seconds or so. I've seen the term "warming" u

Re: Searching on a Rapidly changing Index

2007-05-24 Thread Erick Erickson
Yep. You probably want to do some sorting by other than relevancy too in order to fill the sort caches. Erick On 5/24/07, Joe Shaw <[EMAIL PROTECTED]> wrote: Hi, On 5/24/07, Erick Erickson <[EMAIL PROTECTED]> wrote: > If that's unacceptable, you can *still* open up a new reader in the > b

Re: Searching on a Rapidly changing Index

2007-05-24 Thread Joe Shaw
Hi, On 5/24/07, Erick Erickson <[EMAIL PROTECTED]> wrote: If that's unacceptable, you can *still* open up a new reader in the background and warm it up before using it. "immediately" then becomes 5-10 seconds or so. I've seen the term "warming" used a few times on the various lists. What const

Re: Searching on a Rapidly changing Index

2007-05-24 Thread Simon Wistow
On Thu, May 24, 2007 at 09:28:30AM -0400, Erick Erickson said: > If that's unacceptable, you can *still* open up a new reader in the > background and warm it up before using it. "immediately" then > becomes 5-10 seconds or so. This is currently what I'm doing using a list of previous performed qu

Re: Searching on a Rapidly changing Index

2007-05-24 Thread Erick Erickson
Another option would be to only re-open your searcher when actually needed, that is after the index has changed. This only does you some good when you have some hope that there are sizable gaps in your modifications Another possibility is to relax the "immediately" constraint. Would a maximum

RE: Searching on a Rapidly changing Index

2007-05-24 Thread Mordo, Aviran (EXP N-NANNATEK)
viran http://www.aviransplace.com http://shaveh.co.il -Original Message- From: Simon Wistow [mailto:[EMAIL PROTECTED] Sent: Thursday, May 24, 2007 5:22 AM To: Lucene Subject: Searching on a Rapidly changing Index I've built a Lucene system that gets rapidly updated - documents are s

Searching on a Rapidly changing Index

2007-05-24 Thread Simon Wistow
I've built a Lucene system that gets rapidly updated - documents are supposed to be searchable immeidately after they've been indexed. As such I have a Writer that puts new index, update and delete tasks into a queue and then has a thread which consumes them and applies them to the index using