Re: inserting millions of entries

2007-06-28 Thread Erick Erickson
Yes, opening/closing will be very costly. But I *believe*, although I haven't tried it, that IndexModifier (2.1) will work for you. But do NOT take my word for it as I haven't tried to do what you're doing. But it should be easy to write a short test or two to prove that you can find recently-ins

Re: inserting millions of entries

2007-06-28 Thread Mathieu Lecarme
stop writing scp index to another computer play with it scp indexModified to the server mv indexModified indexCurrent all done. mv is atomic. Jens Grivolla a écrit : > Hi, > > I have a Lucene index with a few million entries, and I will need to > add batches of a few hundred thousand or a few mil

inserting millions of entries

2007-06-28 Thread Jens Grivolla
Hi, I have a Lucene index with a few million entries, and I will need to add batches of a few hundred thousand or a few million additional entries. Unfortunately, I absolutely need to have all indexed entries available when inserting a new one, even within one batch, in order to do some duplicat