We're using a single dual-3Ghz Xeon box, Sun vx65 - indexes stored on Netapp nearstore R100. I think you can either try to investigate if there's a way your users will naturally group their searches and build indexes around that to minimize individual index size or prototype a distributed index search across enough processors to meet your service times.
Hope that datapoint helps.... Vince On 5/16/05, Richard Krenek <[EMAIL PROTECTED]> wrote: > > Unfortunately our indexes will be performance sensitive. Is Lucene > still a good choice? What kind of hardware are you using? > > Also what are the performance implications for having the additional > 80 records in the index for just display purposes? > > Thanks, > Richard Krenek > > > On 5/13/05, Vince Taluskie <[EMAIL PROTECTED]> wrote: > > Yes, you'll be fine with 100 million, I've got a couple of > non-performance > > sensitive indexes that are more than double that (280M) with about 20 > > seachable fields as well. We get results back in the 10-20 second range > > which is fine for our end users. > > > > Vince > > > > > > On 5/13/05, Richard Krenek <[EMAIL PROTECTED]> wrote: > > > > > > Hypothetically I have 100 million records. Each record has 100+ > > > fields. Only 20 of those fields need to be searched on, the rest > > > (including the 20) are just for display purposes. > > > Would it be best to just add the 20 fields to the index and keep the > > > rest in a relational database? What affect does all that fluff data > > > have on the index size and search speeds? Does it matter that some of > > > the fluff data is repeated a lot. (certain fields might just contain > > > state a person lives, the color of their hair, number of fingers, > > > etc). > > > Our indexes are going to be very big, 100 million+ is not an > > > exageration. Will Lucene handle this ok? I have created indexes in the > > > 8-30 million range, but never this big in the number of documents and > > > also the number of fields. > > > > > > Thanks for any info you can provide. > > > > > > > > --------------------------------------------------------------------- > > > To unsubscribe, e-mail: > > [EMAIL PROTECTED] > > > For additional commands, e-mail: > > [EMAIL PROTECTED] > > > > > > > > > > > > > > -- > > > > @work > > @home > > > > vince.taluskie (at) cexp.com <http://cexp.com> vince (at) > > taluskie.com<http://taluskie.com> > > Corporate Express; Technical Architect Louisville, CO > > Phone: 303 664 2660 > > http://www.taluskie.com > > > -- @work @home vince.taluskie (at) cexp.com <http://cexp.com> vince (at) taluskie.com<http://taluskie.com> Corporate Express; Technical Architect Louisville, CO Phone: 303 664 2660 http://www.taluskie.com
