Re: best practices for simulating transactions in Cassandra

2011-12-15 Thread John Laban
I'm actually using Curator as a Zookeeper client myself. I haven't used it in production yet, but so far it seems well written and Jordan Zimmerman at Netflix has been great on the support end as well. I haven't tried Cages so I can't really compare, but I think one of the main deciding factors b

Re: best practices for simulating transactions in Cassandra

2011-12-15 Thread Boris Yen
I am not sure if this is the right thread to ask about this. I read that some people are using cage+zookeeper. I was wondering if anyone evaluates https://github.com/Netflix/curator? this seems to be a versatile package. On Tue, Dec 13, 2011 at 6:06 AM, John Laban wrote: > Ok, great. I'll be s

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread John Laban
Ok, great. I'll be sure to look into the virtualization-specific NTP guides. Another benefit of using Cassandra over Zookeeper for locking is that you don't have to worry about losing your connection to Zookeeper (and with it your locks) while hammering away at data in Cassandra. If using Cassan

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread John Laban
Hey Jake, So I guess my problem is that I've never really relied on NTP before to try to guarantee consistency in my application. Does it tend to work really well in practice? What's the maximum clock skew you can see even when running NTP (especially if you're using more than one DC where you m

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread Dominic Williams
Hi John, On 12 December 2011 19:35, John Laban wrote: > > So I responded to your algorithm in another part of this thread (very > interesting) but this part of the paper caught my attention: > > > When client application code releases a lock, that lock must not > actually be > > released for a pe

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread Jake Luciani
> > > Jake: The algorithm you've outlined is pretty similar to how Zookeeper > clients implement locking. The potential only issue that I see with it > implemented in Cassandra is that it uses the timestamps of the inserted > columns to determine the winner of the lock. The column timestamps are

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread John Laban
Hi Dominic, So I responded to your algorithm in another part of this thread (very interesting) but this part of the paper caught my attention: > When client application code releases a lock, that lock must not actually be > released for a period equal to one millisecond plus twice the maximum pos

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread John Laban
Dominic/Jake: very interesting. This is getting more into fundamentals on locking/isolation rather than transactions/atomicity, but it is still relevant as I was going to use ZooKeeper for that stuff, but it would certainly nice to KISS and remove a component from my setup if I can do without it.

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread Jake Luciani
I've written a locking mechanism for Solandra (I refer to it as a reservation system) which basically allows you to acquire a lock. This is used to ensure a node is service unique sequential IDs for lucene. It sounds a bit similar to Dominic's description but I'll explain how the Solandra one wo

Re: best practices for simulating transactions in Cassandra

2011-12-12 Thread Dominic Williams
Hi guys, just thought I'd chip in... Fight My Monster is still using Cages, which is working fine, but... I'm looking at using Cassandra to replace Cages/ZooKeeper(!) There are 2 main reasons:- 1. Although a fast ZooKeeper cluster can handle a lot of load (we aren't getting anywhere near to capa

Re: best practices for simulating transactions in Cassandra

2011-12-10 Thread Guy Incognito
you could try writing with the clock of the initial replay entry? On 06/12/2011 20:26, John Laban wrote: Ah, neat. It is similar to what was proposed in (4) above with adding transactions to Cages, but instead of snapshotting the data to be rolled back (the "before" data), you snapshot the dat

Re: best practices for simulating transactions in Cassandra

2011-12-06 Thread Jérémy SEVELLEC
I dont' use exactly the same approach describe in the presentation. I use the idea to write whole data in one time in one column value to have atomicity. My application make a read before write... to verify the state of the last transactionlog. If there is data in my "transactionlog" column, it' b

Re: best practices for simulating transactions in Cassandra

2011-12-06 Thread John Laban
Ah, neat. It is similar to what was proposed in (4) above with adding transactions to Cages, but instead of snapshotting the data to be rolled back (the "before" data), you snapshot the data to be replayed (the "after" data). And then later, if you find that the transaction didn't complete, you j

Re: best practices for simulating transactions in Cassandra

2011-12-06 Thread Jérémy SEVELLEC
Hi John, I had exactly the same reflexions. I'm using zookeeper and cage to lock et isolate. but how to rollback? It's impossible so try replay! the idea is explained in this presentation http://www.slideshare.net/mattdennis/cassandra-data-modeling (starting from slide 24) - insert your whole

best practices for simulating transactions in Cassandra

2011-12-05 Thread John Laban
Hello, I'm building a system using Cassandra as a datastore and I have a few places where I am need of transactions. I'm using ZooKeeper to provide locking when I'm in need of some concurrency control or isolation, so that solves that half of the puzzle. What I need now is to sometimes be able t