I used the following settings for speeding up indexing on a similarly sized db table If you have enough ram it might help you.
IndexWriter writer = *new* IndexWriter(fdDir,*new* StandardAnalyzer(), *true *); writer.setMergeFactor(100); writer.setMaxMergeDocs(999999); writer.setMaxBufferedDocs(1500); On 1/11/07, Rangarirayi Muvavarirwa <[EMAIL PROTECTED]> wrote:
One option: https://cool-apps-distributedindex.dev.java.net/ caveat: you would have to setup an account (you get 10CPUhr & 10GB account upon signup) On 1/11/07, Alice <[EMAIL PROTECTED]> wrote: > > Unfortunately I can't use multiple machines. > > And I cannot start lots of threads because the server crashes. > > -----Original Message----- > From: Russ [mailto:[EMAIL PROTECTED] > Sent: quinta-feira, 11 de janeiro de 2007 14:33 > To: java-user@lucene.apache.org > Subject: Re: Huge Index > > Can you use multiple threads/machines to index the data into separate > indexes, and then combine them? > > Russ > Sent wirelessly via BlackBerry from T-Mobile. > > -----Original Message----- > From: "Alice" <[EMAIL PROTECTED]> > Date: Thu, 11 Jan 2007 13:47:36 > To:<java-user@lucene.apache.org> > Subject: Huge Index > > Hello! > > > > I have to index 37million documents retrieved from the database. > > > > I was trying to do by loading intervals of 10000 records but it is too > slow. > > > > Anybody could sugest a better way to get all the data indexed in a > reasonable time? > > > > Thanks > > > > > > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
