Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-04 Thread Kumanan Rajamanikkam
gt; Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Kumanan [mailto:kuma...@gmail.com] > > Sent: Sunday, January 03, 2010 4:24 AM > > To: java-

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Yonik Seeley
On Sun, Jan 3, 2010 at 10:42 AM, Karl Wettin wrote: > > 3 jan 2010 kl. 16.32 skrev Yonik Seeley: > >> Perhaps this is just a huge index, and not enough of it can be cached in >> RAM. >> Adding additional clauses to a boolean query incrementally destroys >> locality. >> >> 104GB of index and 4GB of

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Karl Wettin
3 jan 2010 kl. 16.32 skrev Yonik Seeley: Perhaps this is just a huge index, and not enough of it can be cached in RAM. Adding additional clauses to a boolean query incrementally destroys locality. 104GB of index and 4GB of RAM means you're going to be hitting the disk constantly. You need

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Yonik Seeley
Perhaps this is just a huge index, and not enough of it can be cached in RAM. Adding additional clauses to a boolean query incrementally destroys locality. 104GB of index and 4GB of RAM means you're going to be hitting the disk constantly. You need more hardware - if you're requirements are low (

RE: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Uwe Schindler
ery performance with 1/2 billion documents in > the index > > Uwe, > > Thank you for your response. > > Here is some more information. > > CPU - We use 2 processor Quad Core intel CPU. (not sure about the > particular > model. I will find out) > > JVM -

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-02 Thread Kumanan
: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Saturday, January 02, 2010 10:46 PM > > To: java-user@lucene.apache.org; kuma...@kumanan.com > > Subject: RE: NumericRangeQuery performance with 1/2 billion documents in > > the index > > > > The information you

RE: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-02 Thread Uwe Schindler
age- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Saturday, January 02, 2010 10:46 PM > To: java-user@lucene.apache.org; kuma...@kumanan.com > Subject: RE: NumericRangeQuery performance with 1/2 billion documents in > the index > > The information you gave us is a littl

RE: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-02 Thread Uwe Schindler
The information you gave us is a little spare. - What JVM do you use, what processor,... - How many documents match the query? NRQ is very fast, but if your range hits e.g. one third of all documents, the hit collection of 166 mill docs also takes lots of time. 7 seconds is normal for this case. Ev