On 06/04/2010 18:53, Tatu Saloranta wrote: >> I've read all about QUORUM, and it is generally useful, but as far as I >> can tell, it can't give me a transaction... >> > Correct. Only individual operations are atomic, and ordering of > insertions is not guaranteed. > As I thought. > I think there were some logged Jira issues to allow grouping of > operations into what seems to amount to transactions, which could help > a lot here... but I can't find it now (or maybe it has only been > discussed so far?). > If I understand this correctly, it would just mean that you could send > a sequence of operations, to be completed as a unit (first into > journal, then into memtable etc). >
I think we're on the same page. I need an atomic 'transaction' affecting multiple keys - so I write a tuple of all the updates (inserts/deletes) as a single value into a 'merge-pending' keyset... and (somehow - perhaps with memtable) I modify data read from other keysets to be as-if this 'merge-pending' data had already been been applied to the independent keysets to which it relates. A process/thread on each node would continuously attempt to apply the multiple updates from the merge-pending data before deleting it and dropping the associated merge-data from the in-memory transformations. Latency should be very low (like with a log-based file-system) and throughput should be reasonably high because there should be a lot of flexibility in batch processing the 'merge-pending' data. This way, if there's a failure during merging, there's sufficient durable record to complete the merge before serving any more remote requests. To the remote client, it appears indistinguishable from an atomic transaction that affected more than one key.
