Hi, That mergeFactor is too high. I suggest going back to default (10). maxBufferedDocs is an old and not very accurate setting (imagine what happens with the JVM heap if your indexer hits a SUPER LARGE document). Use setRamBufferSizeMB instead.
Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR ----- Original Message ---- > From: Laxmilal Menariya <lmena...@chambal.com> > To: java-user@lucene.apache.org > Sent: Monday, August 10, 2009 3:23:17 AM > Subject: Taking too much time in optimization > > Hello everyone, > > I have created a sample application & indexing files properties, have index > appx 107K files. > > I am getting OutofMemoryError after 100K while indexing, got the cause from > MaxBuffereddocs=100K, but after that I am calling optimize() method, this is > taking too much time appx 12-HRS, and index size is more than 500GB, its too > large. > > I am using Lucene 2.4.0. Could some one please let me know what wrong with > my configuration. > > My Configuration is : > > lucWriter = new IndexWriter("C:\\Laxmilal", new KeywordAnalyzer(), > true); > lucWriter.setMergeFactor((int) 1000); > lucWriter.setMaxMergeDocs((int) 2147483647); > lucWriter.setMaxBufferedDocs((int) 100000); > > > -- > Thanks, > Laxmilal Menariya > > http://www.bucketexplorer.com/ > http://www.sdbexplorer.com/ > http://www.chambal.com/ --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
