RE: Lucene slow performance -- still broke

2013-03-20 Thread Scott Smith
Duh...it's supposed to be setMergeFactor(). Thanks Scott -Original Message- From: Simon Willnauer [mailto:simon.willna...@gmail.com] Sent: Wednesday, March 20, 2013 3:53 PM To: java-user@lucene.apache.org Subject: Re: Lucene slow performance -- still broke quick question, w

Re: Lucene slow performance -- still broke

2013-03-20 Thread Simon Willnauer
.setRAMBufferSizeMB(50.0); > > Any help in figuring out what is causing this problem would be appreciated. > I do now have an offline system that I can play with so I can do some > intrusive things if need be. > > Scott > > > > > -Origi

RE: Lucene slow performance -- still broke

2013-03-20 Thread Scott Smith
Saturday, March 16, 2013 1:28 PM To: java-user@lucene.apache.org Subject: RE: Lucene slow performance Thanks for the help. The reindex was done this morning and searches now take less than a second. I will make the change to the code. Cheers Scott -Original Message- From: Uwe Schindle

RE: Lucene slow performance

2013-03-16 Thread Scott Smith
Subject: RE: Lucene slow performance Please forceMerge only one time not every time (only to clean up your index)! If you are doing a reindex already, just fix your close logic as discussed before. Scott Smith schrieb: >Unfortunately, this is a production system which I can't touch (th

RE: Lucene slow performance

2013-03-15 Thread Uwe Schindler
riday, March 15, 2013 5:08 PM >To: java-user@lucene.apache.org >Subject: Re: Lucene slow performance > >On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith > wrote: >> " Do you always close IndexWriter after adding few documents and when >closing, disable "wait for merge"? I

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
nauer [mailto:simon.willna...@gmail.com] Sent: Friday, March 15, 2013 5:08 PM To: java-user@lucene.apache.org Subject: Re: Lucene slow performance On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith wrote: > " Do you always close IndexWriter after adding few documents and when > closing, d

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
March 16, 2013 12:08 AM > To: java-user@lucene.apache.org > Subject: Re: Lucene slow performance > > On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith > wrote: > > " Do you always close IndexWriter after adding few documents and > > when > closing, disable "

RE: Lucene slow performance

2013-03-15 Thread Uwe Schindler
o: java-user@lucene.apache.org > Subject: Re: Lucene slow performance > > On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith > wrote: > > " Do you always close IndexWriter after adding few documents and when > closing, disable "wait for merge"? In that case, all

Re: Lucene slow performance

2013-03-15 Thread Simon Willnauer
ry odd though, do you see file that get actually removed / merged if you call IndexWriter#forceMerge(1) simon > > Thanks > > Scott > > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Friday, March 15, 2013 4:49 PM > To: java-user@lucene.apache

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
java-user@lucene.apache.org Subject: RE: Lucene slow performance Hi, with standard configuartion, this cannot happen. What merge policy do you use? This looks to me like a misconfigured merge policy or using the NoMergePolicy. With 3,000 segments, it will be slow, the question is, why do you get

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
auer [mailto:simon.willna...@gmail.com] Sent: Friday, March 15, 2013 4:45 PM To: java-user@lucene.apache.org Subject: Re: Lucene slow performance Can you tell us a little more about how you use lucene, how do you index, do you use NRT or do you open an IndexReader for every request, do you maybe us

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
A little more data, of the 3330 files in the index, 2173 are CFS files and average 120k. Another 1116 files are .del's and average about 4kB. The remaining .prx, .frq, etc. consists of 41 files and total only 101MB. The largest files are 3 .prx files which total less than 60MB and 2 .frq of a

RE: Lucene slow performance

2013-03-15 Thread Uwe Schindler
Hi, with standard configuartion, this cannot happen. What merge policy do you use? This looks to me like a misconfigured merge policy or using the NoMergePolicy. With 3,000 segments, it will be slow, the question is, why do you get those? Another thing could be: Do you always close IndexWriter

Re: Lucene slow performance

2013-03-15 Thread Simon Willnauer
Can you tell us a little more about how you use lucene, how do you index, do you use NRT or do you open an IndexReader for every request, do you maybe us a custom merge policy or somthing like this, any special IndexWriter settings? On Fri, Mar 15, 2013 at 11:15 PM, Scott Smith wrote: > We have a