Hi Trevor, I hope to post on my practical experiences in this area soon - we rely heavily on complex serialized operations in FightMyMonster.com. Probably the most simple serialized operation we do is updating nugget balances when, for example, there has been a trade of monsters.
Currently we use ZooKeeper/Cages (github.com/s7) to serialize our distributed ops. We don't implement transactions with rollback/commit. Rather, we lock some paths, for example /Users/bank/dominic and /Users/bank/ben, and then write with QUORUM through our Java client library Pelops. This will make several efforts to retry the operation if it fails at first, and in our line of business the fact that redundancy in the cluster means it will nearly always complete eventually is enough. Of course, in a real world money scenario that is not enough and data inconsistency caused by, say, a sudden power outage during the retry phase is not acceptable. To handle this case I would like to extend Cages at some point so that commit/rollback transactions that would be stored inside ZooKeeper are associated with the distributed locks (which are stored persistently and survive power loss for example). There is an old blog post here which talks about it http://ria101.wordpress.com/2010/05/12/locking-and-transactions-over-cassandra-using-cages/although this needs updating. One interesting point not discussed which I have also not heard mentioned elsewhere is that in order for serialization to work every time, before you release a lock after performing an update you must wait for a brief period >= max variance between the clocks on the application nodes updating the database e.g. 1-2ms. This is because Cassandra uses the timestamps of columns that have been written during reconciliation to determine which should be persisted when they conflict. As far as scaling goes, ZooKeeper can be scaled by having several clusters and hashing lock paths to them. Alternatively, Lamport's bakery algorithm could be investigated as this shows you can have locking without a central coordinator service. Best, Dominic On 22 June 2011 15:18, Trevor Smith <tre...@knewton.com> wrote: > Hello, > > I was wondering if anyone had architecture thoughts of creating a simple > bank account program that does not use transactions. I think creating an > example project like this would be a good thing to have for a lot of the > discussions that pop up about transactions and Cassandra (and > non-transactional datastores in general). > > Consider the simple system that has accounts, and users can transfer money > between the accounts. > > There are these interesting papers as background (links below). > > Thank you. > > Trevor Smith > > http://www.ics.uci.edu/~cs223/papers/cidr07p15.pdf > > > http://blogs.msdn.com/cfs-file.ashx/__key/communityserver-components-postattachments/00-09-20-52-14/BuildingOnQuicksand_2D00_V3_2D00_081212h_2D00_pdf.pdf > > http://www.cidrdb.org/cidr2011/Papers/CIDR11_Paper32.pdf >
