Re: Lucene index write performance optimization

2009-11-10 Thread Otis Gospodnetic
This is what we have in Lucene in Action 2: ~/lia2$ ff \*Thread\*java ./src/lia/admin/CreateThreadedIndexTask.java ./src/lia/admin/ThreadedIndexWriter.java Otis -- Sematext is hiring -- http://sematext.com/about/jobs.html?mls Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR -

Re: Lucene index write performance optimization

2009-11-10 Thread Yonik Seeley
On Tue, Nov 10, 2009 at 11:43 AM, Jamie Band wrote: > As an aside note, is there any way for Lucene to support simultaneous writes > to an index? The indexing process is highly parallelized... just use multiple threads to add documents to the same IndexWriter. -Yonik http://www.lucidimagination.

Re: Lucene index write performance optimization

2009-11-10 Thread Glen Newton
You might try re-implementing, using ThreadPoolExecutor http://java.sun.com/j2se/1.5.0/docs/api/java/util/concurrent/ThreadPoolExecutor.html glen 2009/11/10 Jamie Band : > Hi There > > Our app spends alot of time waiting for Lucene to finish writing to the > index. I'd like to minimize this. If y