I'll run some tests. Thank you. > From: [EMAIL PROTECTED] > To: java-user@lucene.apache.org > Subject: Re: Index optimization ... > Date: Wed, 30 Jul 2008 11:12:28 -0400 > > What version of Lucene are you using? What is your current > mergeFactor? Lowering this (minimum is 2) will result in an index > that is closer to "optimal" since an optimized index is just one that > has all the segments merged into a single segment and a mergeFactor of > 2 just means there are only ever 2 segments (the docs could make this > clearer). The tradeoff is that you may need to do merges more often. > If you are using Lucene 2.3.x these merges can now take place in the > background, so this may not be as big a penalty as it once was. > Still, the fact is, optimize has to go through, in the end, and merge > your segments into one big segment. This is a lengthy undertaking on > a large index. > > I'm not sure, however, if any of this will reduce your overall time. > I suppose it depends somewhat on your update rate. It is possible > that the slower indexing could offset any gains had by a faster > optimize. Another option may be to keep the mergeFactor higher, but > then every so often do partial optimizes to keep your index closer to > optimal such that a final optimize may be speed up > > Another question is have you measured your query performance on an > unoptimized index? Is it acceptable? Are you only adding new > documents or are you also deleting docs in that 4 hour time period? > > Bottom line, though, is you need to test out the various knobs > (mergeFactor, RAMBufferSizeMB, etc.) and see. You may find the > contrib/benchmark program helpful for running experiments, although it > isn't a substitute for your actual data. > > -Grant > > > On Jul 30, 2008, at 9:46 AM, Dragon Fly wrote: > > > Perhaps I didn't explain myself clearly so please let me try it > > again. I'm happy with the search/indexing performance. However, > > my index gets fully optimized every 4 hours and the time it takes to > > fully optimize the index is longer than I like. Is there anything > > that I can do to speed up the optimization? I don't fully understand > > the different parameters (e.g. merge factor). If I decrease the > > merge factor, would it make the indexing slower (which I'm OK with) > > but the optimization faster? Thank you. > > > >> Date: Tue, 29 Jul 2008 08:32:46 +0200 > >> From: [EMAIL PROTECTED] > >> To: java-user@lucene.apache.org > >> Subject: Re: Index optimization ... > >> > >> John Griffin: > >>> Use IndexWriter.setRAMBufferSizeMB(double mb) and you won't have to > >>> sacrifice anything. It defaults to 16.0 MB so depending on the > >>> size of your > >>> index you may want to make it larger. Do some testing at various > >>> values to > >>> see where the sweet spot is. > >>> > >> > >> Also, have a look at > >> http://wiki.apache.org/lucene-java/ImproveIndexingSpeed, which > >> provides > >> a range of helping advices in terms of enhanced indexing speed. > >> > >> -- > >> Asbjørn A. Fellinghaug > >> [EMAIL PROTECTED] > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: [EMAIL PROTECTED] > >> For additional commands, e-mail: [EMAIL PROTECTED] > >> > > > > _________________________________________________________________ > > With Windows Live for mobile, your contacts travel with you. > > http://www.windowslive.com/mobile/overview.html?ocid=TXT_TAGLM_WL_mobile_072008 > > -------------------------- > Grant Ingersoll > http://www.lucidimagination.com > > Lucene Helpful Hints: > http://wiki.apache.org/lucene-java/BasicsOfPerformance > http://wiki.apache.org/lucene-java/LuceneFAQ > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] >
_________________________________________________________________ Use video conversation to talk face-to-face with Windows Live Messenger. http://www.windowslive.com/messenger/connect_your_way.html?ocid=TXT_TAGLM_WL_Refresh_messenger_video_072008
