Re: Indexing questions

2008-07-15 Thread Michael McCandless
[EMAIL PROTECTED] wrote: This isn't quite true. If you open IndexWriter with autoCommit=false, then none of the changes you do with it will be visible to an IndexReader, even one reopened while IndexWriter is doing its work, until you close the IndexWriter. Where are the docs for this tran

Re: Indexing questions

2008-07-15 Thread spring
> This isn't quite true. If you open IndexWriter with autoCommit=false, > then none of the changes you do with it will be visible to an > IndexReader, even one reopened while IndexWriter is doing its work, > until you close the IndexWriter. Where are the docs for this transaction buffered?

Re: Indexing questions

2008-07-15 Thread spring
> How about just copying and performing your indexing (or index write > related) > operations on the copy and then performing a rename operation followed by > reopening of the index readers. This is how we did it until now. But the indexes become bigger and bigger (50 GB and more) and so we are

Re: Indexing questions

2008-07-15 Thread Michael McCandless
Anshum wrote: But the downside to this would be, in case your daemon crashes in the meantime or you need to restart the daemon, the index would not be usable until you have completed your indexing processs. This isn't quite true. If you open IndexWriter with autoCommit=false, then none

Re: Indexing questions

2008-07-14 Thread Anshum
Hi, As per my knowledge, you may do any of the below processes while searching (n parallel) just that the changes would not reflect until you reopen the index readers (by either using the reopen command or closing and opening them explicitly). But the downside to this would be, in case your daemon

Re: Indexing questions

2008-07-14 Thread Michael McCandless
The answer to all 3 is yes, but, you'll have to re-open your IndexReader to see any of those changes. An IndexReader always searches the "point in time" snapshot of the index as of the moment it was opened. Any & all changes done with an IndexWriter (including opening a new index in the