Re: realtime indexing

2007-11-16 Thread Kay Röpke
On Nov 16, 2007, at 11:59 AM, Antoine Baudoux wrote: I'm trying to implement a similar solution. Could you be more precise on how you handle duplicates, as well as document deletion? The key probably is (it was for us, anyway) that you have a fast way of determining whether or

Re: realtime indexing

2007-11-16 Thread John Wang
Thanks Kay. I am doing exactly what you are saying. Just to elaborate: So whatever is submitted to the RAM index is always the latest, any deletes (an update is a delete + an add) submitted to the any of the ram indexes is recorded (discarded when the ram index is discarded) with the uid. That

Re: realtime indexing

2007-11-16 Thread Antoine Baudoux
Hi, I'm trying to implement a similar solution. Could you be more precise on how you handle duplicates, as well as document deletion? Thx, Antoine On Nov 16, 2007, at 7:44 AM, John Wang wrote: Hi: It was interesting hearing about the need for real time ind

realtime indexing

2007-11-15 Thread John Wang
Hi: It was interesting hearing about the need for real time indexing at the BirdsOfAFeather round table. We also needed to solve this problem. We took this approach: A large disk index that indexes in batch, e.g. sleeps for some time queue up requests, wakes up and the index. While large disk