Re: Efficient delete

2008-08-16 Thread Michael McCandless
It saves you the hassle of closing your IndexWriter, opening an IndexReader, doing deletes, closing the IndexReader, then opening a new IndexWriter. Also, it can be more efficient (depends on your application) since it may buffer the deletes for longer than you would if you used IndexReader. Mike

Re: Efficient delete

2008-08-16 Thread Michael Zehrer
Mike, what is the difference if I would use IndexWriter? Michael On Fri, Aug 8, 2008 at 12:27, Michael McCandless <[EMAIL PROTECTED]> wrote: > > Also, can you do your deletes via IndexWriter (delete by Term) instead of > opening IndexReader to do the deletes? > > Mike > > Ian Lea wrote: > >> Mic

Re: Efficient delete

2008-08-08 Thread Michael McCandless
Also, can you do your deletes via IndexWriter (delete by Term) instead of opening IndexReader to do the deletes? Mike Ian Lea wrote: Michael Did you get anywhere with this? 3 secs for one delete is excessive. A job of mine ran earlier today and did 2000+ deletes by term on unique id in l

Re: Efficient delete

2008-08-08 Thread Ian Lea
Michael Did you get anywhere with this? 3 secs for one delete is excessive. A job of mine ran earlier today and did 2000+ deletes by term on unique id in less than 9 seconds. The index is smaller, at around 5Gb, but I don't believe that would explain the difference. All the deletes were done in

Efficient delete

2008-08-07 Thread Michael Zehrer
Dear List, I have a rather big index around 20gb. My documents have a unique id that I store in in an untokenized field. Using an IndexReader I delete documents by term using the id. The applications tries to batch as many delete operations as possible for this. The applications runs on a 8Core