RE: Lucene slow performance -- still broke

2013-03-20 Thread Scott Smith
Duh...it's supposed to be setMergeFactor(). Thanks Scott -Original Message- From: Simon Willnauer [mailto:simon.willna...@gmail.com] Sent: Wednesday, March 20, 2013 3:53 PM To: java-user@lucene.apache.org Subject: Re: Lucene slow performance -- still broke quick question, w

Re: Lucene slow performance -- still broke

2013-03-20 Thread Simon Willnauer
.setRAMBufferSizeMB(50.0); > > Any help in figuring out what is causing this problem would be appreciated. > I do now have an offline system that I can play with so I can do some > intrusive things if need be. > > Scott > > > > > -Origi

RE: Lucene slow performance -- still broke

2013-03-20 Thread Scott Smith
Saturday, March 16, 2013 1:28 PM To: java-user@lucene.apache.org Subject: RE: Lucene slow performance Thanks for the help. The reindex was done this morning and searches now take less than a second. I will make the change to the code. Cheers Scott -Original Message- From: Uwe Schindle

RE: Lucene slow performance

2013-03-16 Thread Scott Smith
Subject: RE: Lucene slow performance Please forceMerge only one time not every time (only to clean up your index)! If you are doing a reindex already, just fix your close logic as discussed before. Scott Smith schrieb: >Unfortunately, this is a production system which I can't touch (th

RE: Lucene slow performance

2013-03-15 Thread Uwe Schindler
riday, March 15, 2013 5:08 PM >To: java-user@lucene.apache.org >Subject: Re: Lucene slow performance > >On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith > wrote: >> " Do you always close IndexWriter after adding few documents and when >closing, disable "wait for merge"? I

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
nauer [mailto:simon.willna...@gmail.com] Sent: Friday, March 15, 2013 5:08 PM To: java-user@lucene.apache.org Subject: Re: Lucene slow performance On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith wrote: > " Do you always close IndexWriter after adding few documents and when > closing, d

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
March 16, 2013 12:08 AM > To: java-user@lucene.apache.org > Subject: Re: Lucene slow performance > > On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith > wrote: > > " Do you always close IndexWriter after adding few documents and > > when > closing, disable "

RE: Lucene slow performance

2013-03-15 Thread Uwe Schindler
o: java-user@lucene.apache.org > Subject: Re: Lucene slow performance > > On Sat, Mar 16, 2013 at 12:02 AM, Scott Smith > wrote: > > " Do you always close IndexWriter after adding few documents and when > closing, disable "wait for merge"? In that case, all

Re: Lucene slow performance

2013-03-15 Thread Simon Willnauer
ry odd though, do you see file that get actually removed / merged if you call IndexWriter#forceMerge(1) simon > > Thanks > > Scott > > -Original Message- > From: Uwe Schindler [mailto:u...@thetaphi.de] > Sent: Friday, March 15, 2013 4:49 PM > To: java-user@lucene.apache

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
java-user@lucene.apache.org Subject: RE: Lucene slow performance Hi, with standard configuartion, this cannot happen. What merge policy do you use? This looks to me like a misconfigured merge policy or using the NoMergePolicy. With 3,000 segments, it will be slow, the question is, why do you get

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
auer [mailto:simon.willna...@gmail.com] Sent: Friday, March 15, 2013 4:45 PM To: java-user@lucene.apache.org Subject: Re: Lucene slow performance Can you tell us a little more about how you use lucene, how do you index, do you use NRT or do you open an IndexReader for every request, do you maybe us

RE: Lucene slow performance

2013-03-15 Thread Scott Smith
n has changed since 1.4, but does it not merge all of the various files into a few files? -Original Message- From: Scott Smith [mailto:ssm...@mainstreamdata.com] Sent: Friday, March 15, 2013 4:15 PM To: java-user@lucene.apache.org Subject: Lucene slow performance We have a system th

RE: Lucene slow performance

2013-03-15 Thread Uwe Schindler
ndler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Scott Smith [mailto:ssm...@mainstreamdata.com] > Sent: Friday, March 15, 2013 11:15 PM > To: java-user@lucene.apache.org > Subject: Lucene slow performance >

Re: Lucene slow performance

2013-03-15 Thread Simon Willnauer
Can you tell us a little more about how you use lucene, how do you index, do you use NRT or do you open an IndexReader for every request, do you maybe us a custom merge policy or somthing like this, any special IndexWriter settings? On Fri, Mar 15, 2013 at 11:15 PM, Scott Smith wrote: > We have a

Lucene slow performance

2013-03-15 Thread Scott Smith
We have a system that is using lucene and the searches are very slow. The number of documents is fairly small (less than 30,000) and each document is typically only 2 to 10 kilo-characters. Yet, searches are taking 15-16 seconds. One of the things I noticed was that the index directory has sev