Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-04 Thread Kumanan Rajamanikkam
gt; Uwe > > - > Uwe Schindler > H.-H.-Meier-Allee 63, D-28213 Bremen > http://www.thetaphi.de > eMail: u...@thetaphi.de > > > -Original Message- > > From: Kumanan [mailto:kuma...@gmail.com] > > Sent: Sunday, January 03, 2010 4:24 AM > > To: java-

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Yonik Seeley
On Sun, Jan 3, 2010 at 10:42 AM, Karl Wettin wrote: > > 3 jan 2010 kl. 16.32 skrev Yonik Seeley: > >> Perhaps this is just a huge index, and not enough of it can be cached in >> RAM. >> Adding additional clauses to a boolean query incrementally destroys >> locality. >> >> 104GB of index and 4GB of

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Karl Wettin
3 jan 2010 kl. 16.32 skrev Yonik Seeley: Perhaps this is just a huge index, and not enough of it can be cached in RAM. Adding additional clauses to a boolean query incrementally destroys locality. 104GB of index and 4GB of RAM means you're going to be hitting the disk constantly. You need

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Yonik Seeley
Perhaps this is just a huge index, and not enough of it can be cached in RAM. Adding additional clauses to a boolean query incrementally destroys locality. 104GB of index and 4GB of RAM means you're going to be hitting the disk constantly. You need more hardware - if you're requirements are low (

RE: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-03 Thread Uwe Schindler
ery performance with 1/2 billion documents in > the index > > Uwe, > > Thank you for your response. > > Here is some more information. > > CPU - We use 2 processor Quad Core intel CPU. (not sure about the > particular > model. I will find out) > > JVM -

Re: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-02 Thread Kumanan
: Uwe Schindler [mailto:u...@thetaphi.de] > > Sent: Saturday, January 02, 2010 10:46 PM > > To: java-user@lucene.apache.org; kuma...@kumanan.com > > Subject: RE: NumericRangeQuery performance with 1/2 billion documents in > > the index > > > > The information you

RE: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-02 Thread Uwe Schindler
age- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Saturday, January 02, 2010 10:46 PM > To: java-user@lucene.apache.org; kuma...@kumanan.com > Subject: RE: NumericRangeQuery performance with 1/2 billion documents in > the index > > The information you gave us is a littl

RE: NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-02 Thread Uwe Schindler
rg > Subject: NumericRangeQuery performance with 1/2 billion documents in the > index > > Hi, > > We have an index with 500 million documents in the index. Index size is > 104 > GB and 4 GB RAM for the search server. > > When we try to do NumericRangeQuery on document_date f

NumericRangeQuery performance with 1/2 billion documents in the index

2010-01-02 Thread Kumanan
Hi, We have an index with 500 million documents in the index. Index size is 104 GB and 4 GB RAM for the search server. When we try to do NumericRangeQuery on document_date field, it takes around 7-10 seconds. Is this expected for this size index? Here is how I index that field. docu