RE: search time & number of segments

2014-05-21 Thread De Simone, Alessandro
i 2014 22:09 To: java-user@lucene.apache.org Subject: RE: search time & number of segments De Simone, Alessandro [alessandro.desim...@bvdinfo.com] wrote: > We have stopped optimizing the index because everybody told us it was a bad > idea. > It makes sense if you think about it. When

RE: search time & number of segments

2014-05-20 Thread Toke Eskildsen
De Simone, Alessandro [alessandro.desim...@bvdinfo.com] wrote: > We have stopped optimizing the index because everybody told us it was a bad > idea. > It makes sense if you think about it. When you reopen the index not all > segments must be reopened then you have: > (1) better reload time >

RE: search time & number of segments

2014-05-20 Thread De Simone, Alessandro
ig impact on performance. -Original Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: mardi 20 mai 2014 15:46 To: java-user@lucene.apache.org Subject: Re: search time & number of segments On Tue, 2014-05-20 at 15:04 +0200, De Simone, Alessandro wrote: Tok

Re: search time & number of segments

2014-05-20 Thread Toke Eskildsen
On Tue, 2014-05-20 at 15:04 +0200, De Simone, Alessandro wrote: Toke: > > Using the calculator, I must admit that it is puzzling that you have > 2432 / 143 = 17.001 times the amount of seeks with 16 segments. > > Do you have any clue? Is there something I could test? If your segmented index was

RE: search time & number of segments

2014-05-20 Thread De Simone, Alessandro
iginal Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: lundi 19 mai 2014 16:43 To: java-user@lucene.apache.org Subject: Re: search time & number of segments On Mon, 2014-05-19 at 11:54 +0200, De Simone, Alessandro wrote: [24GB index, 8GB disk cache, only indexed fields] &

Re: search time & number of segments

2014-05-19 Thread Toke Eskildsen
On Mon, 2014-05-19 at 11:54 +0200, De Simone, Alessandro wrote: [24GB index, 8GB disk cache, only indexed fields] > The "IO calls" I was referring to is the number of time the > "BufferedIndexInput.refill()" function is called. So it means that we > have 16 times more bytes read when there are 16

RE: search time & number of segments

2014-05-19 Thread De Simone, Alessandro
iginal Message- From: Toke Eskildsen [mailto:t...@statsbiblioteket.dk] Sent: samedi 17 mai 2014 20:04 To: java-user@lucene.apache.org Subject: RE: search time & number of segments De Simone, Alessandro [alessandro.desim...@bvdinfo.com] wrote: > We have a performance issue ever since we stopped optimiz

RE: search time & number of segments

2014-05-17 Thread Toke Eskildsen
De Simone, Alessandro [alessandro.desim...@bvdinfo.com] wrote: > We have a performance issue ever since we stopped optimizing the index. We > are using Lucene 4.8 (jvm 32bits for searching, 64bits for indexing) on > Windows 2008R2. How much RAM does your search machine have? > For instance, a s

search time & number of segments

2014-05-16 Thread De Simone, Alessandro
Hello everyone, We have a performance issue ever since we stopped optimizing the index. We are using Lucene 4.8 (jvm 32bits for searching, 64bits for indexing) on Windows 2008R2. Now we are letting Lucene handle the merges using the default merge policy (TieredMergePolicy). We have narrowed do