Re: Searching on a Rapidly changing Index

2007-05-24 Thread Joe Shaw
Hi, On 5/24/07, Erick Erickson <[EMAIL PROTECTED]> wrote: If that's unacceptable, you can *still* open up a new reader in the background and warm it up before using it. "immediately" then becomes 5-10 seconds or so. I've seen the term "warming" used a few times on the various lists. What const

Re: Using ParallelReader over large immutable index and small updatable index

2007-03-07 Thread Joe Shaw
Hi, On Tue, 2007-03-06 at 15:34 -0500, Andy Liu wrote: > Is there a working solution out there that would let me use ParallelReader > to search over a large, immutable index and a smaller, auxillary index that > is updated frequently? Currently, from my understanding, the ParallelReader > fails

Re: Use the Luke, Force

2007-01-11 Thread Joe Shaw
Hi, Benson Margulies wrote: My experience tonight is that the stock 1.9-based Luke won't open my 2.0 indices. So I fixed up a version of the source. I've been seeing this too. Anyone else want it? That would be great, if you don't mind. A jar would be nice too. :) Joe --

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-21 Thread Joe Shaw
Hi, On Thu, 2006-12-21 at 10:21 -0800, Otis Gospodnetic wrote: > Something like dd if=/path/to/index/foo.cfs of=/dev/null > Basically, force the data through the kernel preemptively, so FS caches it. > Run vmstat while doing it, and if the index hasn't been cached by the FS, > you should see a spi

Re: Announcement: Lucene powering Monster job search index (Beta)

2006-10-30 Thread Joe Shaw
Hi Peter, On Fri, 2006-10-27 at 15:29 -0400, Peter Keegan wrote: > Numeric range search is one of Lucene's weak points (performance-wise) so we > have implemented this with a custom HitCollector and an extension to the > Lucene index files that stores the numeric field values for all documents. >

Re: index architectures

2006-10-18 Thread Joe Shaw
Hi, On Wed, 2006-10-18 at 19:05 +1300, Paul Waite wrote: > No they don't want that. They just want a small number. What happens is > they enter some silly query, like searching for all stories with a single > common non-stop-word in them, and with the usual sort criterion of by date > (ie. a field

Mutliple indexes or many small documents?

2006-10-05 Thread Joe Shaw
Hi, I'm in the process of moving away from Lucene-as-the-data-store to using Lucene solely for text indexing and storing a lot of (frequently changing) metadata in a database. At present, we have two indexes which we search. The primary index contains the static data -- data that changes only wh