Yikes! I just read your blog Dominic. Now I'm worried since my app was going to be mostly cloud-based. But, you didn't mention anything about sleeping for 'max clock variance' after making the ntp-related config changes (maybe you haven't had the time to blog it).

I'm curious, do you think the sleep is required even in a non-virtualized environment? Is it only needed when implementing some kind of lock? Does the type of lock make a difference?

Thanks!
aj (the other one)

On 6/28/2011 11:31 AM, Dominic Williams wrote:
Hi, yes you are correct, and this is a potential problem.

IMPORTANT: If you need to serialize writes from your application servers, for example using distributed locking, then before releasing locks you must sleep for a period equal to the maximum variance between the clocks on your application server nodes.

I had a problem with the clocks on my nodes which led to all kinds of problems. There is a slightly out of date post, which may not mentioned the above point, on my experiences here http://ria101.wordpress.com/2011/02/08/cassandra-the-importance-of-system-clocks-avoiding-oom-and-how-to-escape-oom-meltdown/

Hope this helps
Dominic

On 27 June 2011 23:03, A J <[email protected] <mailto:[email protected]>> wrote:

    During writes, the timestamp field in the column is the system-time of
    that node (correct me if that is not the case and the system-time of
    the co-ordinator is what gets applied to all the replicas).
    During reads, the latest write wins.

    What if there is a clock skew ? It could lead to a stale write
    over-riding the actual latest write, just because the clock of that
    node is ahead of the other node. Right ?



Reply via email to