RE: In memory Lucene configuration

2012-07-18 Thread Doron Yaacoby
Thanks for the input. I am not using Solr. Also, my index has a fixed size, I am not going to update it. -Original Message- From: googoo [mailto:liu...@gmail.com] Sent: 18 July 2012 15:21 To: java-user@lucene.apache.org Subject: Re: In memory Lucene configuration Doron, To verify

RE: In memory Lucene configuration

2012-07-18 Thread Doron Yaacoby
essage- > From: Doron Yaacoby [mailto:dor...@gingersoftware.com] > Sent: 16 July 2012 09:43 > To: java-user@lucene.apache.org > Subject: RE: In memory Lucene configuration > > I haven't tried that yet, but it's an option. The reason I'm waiting on this >

RE: In memory Lucene configuration

2012-07-18 Thread Uwe Schindler
Hi, just to clarify: > In additional, i don't think load whole index to memory is good idea. Since the > index size will always increase. > For me, i change lucene code to disable MMapDirectory, since the index size is > bigger and bigger. > And MMapDirectory will call something like c++ share me

Re: In memory Lucene configuration

2012-07-18 Thread Simon Willnauer
> From: Doron Yaacoby [mailto:dor...@gingersoftware.com] > Sent: 16 July 2012 09:43 > To: java-user@lucene.apache.org > Subject: RE: In memory Lucene configuration > > I haven't tried that yet, but it's an option. The reason I'm waiting on this > is that I a

Re: In memory Lucene configuration

2012-07-18 Thread googoo
ze is bigger and bigger. And MMapDirectory will call something like c++ share memory to load whole index to ram. -- View this message in context: http://lucene.472066.n3.nabble.com/In-memory-Lucene-configuration-tp3995075p3995697.html Sent from the Lucene - Java Users mailing list archive at Nabbl

RE: In memory Lucene configuration

2012-07-18 Thread Doron Yaacoby
@lucene.apache.org Subject: RE: In memory Lucene configuration I haven't tried that yet, but it's an option. The reason I'm waiting on this is that I am expecting many concurrent requests to my application anyway, so having multiple search threads per request might not be the best idea

Re: In memory Lucene configuration

2012-07-16 Thread Simon Willnauer
08:26 > To: java-user@lucene.apache.org > Subject: Re: In memory Lucene configuration > > Have you tried sharding your data? Since you have a fast multi-core box, why > not split your indices N-ways, say the smaller one into 4, and the larger > into 8. Then you can have a pool of dedi

RE: In memory Lucene configuration

2012-07-15 Thread Doron Yaacoby
Vitaly Funstein [mailto:vfunst...@gmail.com] Sent: 16 July 2012 08:26 To: java-user@lucene.apache.org Subject: Re: In memory Lucene configuration Have you tried sharding your data? Since you have a fast multi-core box, why not split your indices N-ways, say the smaller one into 4, and the larger into

RE: In memory Lucene configuration

2012-07-15 Thread Doron Yaacoby
] Sent: 15 July 2012 13:40 To: java-user@lucene.apache.org; simon.willna...@gmail.com Subject: RE: In memory Lucene configuration Thanks for the quick input! I ran a few more tests with your suggested configuration (-Xmx1G -Xms1G with MMapDirectory). At the third time I ran the same test I fi

Re: In memory Lucene configuration

2012-07-15 Thread Vitaly Funstein
t; Sent: 15 July 2012 11:56 > To: java-user@lucene.apache.org > Subject: Re: In memory Lucene configuration > > hey there, > > On Sun, Jul 15, 2012 at 10:41 AM, Doron Yaacoby > wrote: >> Hi, I have the following situation: >> >> I have two pretty large indic

RE: In memory Lucene configuration

2012-07-15 Thread Doron Yaacoby
;t mention before that I'm using Lucene 3.5 and Java 1.7. -Original Message- From: Simon Willnauer [mailto:simon.willna...@gmail.com] Sent: 15 July 2012 11:56 To: java-user@lucene.apache.org Subject: Re: In memory Lucene configuration hey there, On Sun, Jul 15, 2012 at 10:41 AM, Doron

Re: In memory Lucene configuration

2012-07-15 Thread Simon Willnauer
hey there, On Sun, Jul 15, 2012 at 10:41 AM, Doron Yaacoby wrote: > Hi, I have the following situation: > > I have two pretty large indices. One consists of about 1 billion documents > (takes ~6GB on disk) and the other has about 2 billion documents (~10GB on > disk). The documents are very sho

In memory Lucene configuration

2012-07-15 Thread Doron Yaacoby
Hi, I have the following situation: I have two pretty large indices. One consists of about 1 billion documents (takes ~6GB on disk) and the other has about 2 billion documents (~10GB on disk). The documents are very short (4-5 terms each in the text field, and one numeric field with a long valu