Re: key garbage collection

2011-11-03 Thread Justin Karneges
On Thursday, November 03, 2011 02:46:39 AM Erik Søe Sørensen wrote: > On 03-11-2011 07:39, Justin Karneges wrote: > > I wonder if anyone else is doing something like this. I also wonder if > > there is a flaw in this design. Since the lack of the cleanup key can > > indicate a successful commit,

Re: key garbage collection

2011-11-03 Thread Justin Karneges
On Thursday, November 03, 2011 12:49:06 AM you wrote: > On Thu, Nov 3, 2011 at 02:39, Justin Karneges wrote: > > So far so good. Now for handling cleanup. Periodically, we scan the > > "cleanup" bucket for keys to process. Since keys only exist in this > > bucket at the moment of a write (they

Re: key garbage collection

2011-11-03 Thread Erik Søe Sørensen
On 03-11-2011 07:39, Justin Karneges wrote: [snip] I wonder if anyone else is doing something like this. I also wonder if there is a flaw in this design. Since the lack of the cleanup key can indicate a successful commit, I wonder if there might be a problem with eventual consistency whereby so

Re: key garbage collection

2011-11-03 Thread Greg Stein
On Thu, Nov 3, 2011 at 02:39, Justin Karneges wrote: >... > Say you have an operation that requires creating two keys, A and B, and you > succeed in creating A but fail in creating B.  How do you delete A after the > fact?  I have two ideas: > > 1) Run periodic MapReduce operations that do full db

key garbage collection

2011-11-02 Thread Justin Karneges
Hi, In the "atomically updating multiple keys" thread I mentioned the issue of dirtying the database with orphan keys and thought that topic was worth starting a new thread. Say you have an operation that requires creating two keys, A and B, and you succeed in creating A but fail in creating B