Re: Indexing large index with Lucene

2006-09-28 Thread Erick Erickson
I'll gladly defer to your experience on this one . I have to admit that my applications don't care much about updating so I'm not very familiar with the behavior there. Erick On 9/28/06, Chris Lu <[EMAIL PROTECTED]> wrote: I like the approach in your second point. But I have doubt on the first

Re: Indexing large index with Lucene

2006-09-28 Thread Chris Lu
I like the approach in your second point. But I have doubt on the first point. For a production level index, usually pretty big, freqent close/reopen the searcher may not be fast enough, especially when you want to cache sorting. It's better to keep the searchers open. But when the indexing proce

Re: Indexing large index with Lucene

2006-09-28 Thread Erick Erickson
Two things come to mind... First, you can freely write to an index while searching it, the search is always available. I'm pretty sure this includes deleting/readding documents. However, you won't be able to search on the changes in your index until you close/reopen the *searcher*. Second, depen

Indexing large index with Lucene

2006-09-28 Thread Eric Louvard
I'm using Lucene since several year. We had to index allways more documents. I'm now trying to optimise the index process with more than 1.000.000 documents and I can see that the performance will decrease when the index size is greater. I would like to know if someone as allready studied this