I have writes to cassandra that are failing, or at least a read shortly
after a write is still getting an old value. I realize Cassandra is
"eventually consistent" but this system is a single CPU single node with
consistency level set to 1, so this seems odd to me.
My setup:
Cassandra 0.6rc1
client: writing using Net::Cassandra::Easy 0.09 (perl thrift interface)
Cluster: single node. Default out of the box setup. Single CPU system
(its a P4 dinosaur). The client library defaults to read and write
consistency of 1 which is the number of nodes so is like "all" in my case.
I have a test that modifies data and then tests to see if they have the
expected value. These tests pass sometimes and sometimes fail part way
through the test suite. This doesn't seem to be predictable, stopping
cassandra, nuking the data directory and restarting doesn't make the
system any more likely to pass. Passing or failing the test one time
doesn't seem to correlate with the pass or failure of the next attempt.
What I'm seeing is old data still being returned to get requests after
the mutate has run.
For example: I modified the "set" method of my program to output the
value being saved, the old value and the new value and I see things like
this when the tests fail:
Expected value was
55, but got
72 instead, previous value was
72 at lib/ET/Storage/Cassandra.pm line 96.
This means the previous value was 72, the modifications that where done
should have resulted in the score being 55 but I got a 72 instead. Any
ideas what could cause this?
As an aside I motified some other code to use Net::Cassandra instead of
Net::Cassandra::Easy and noticed that it seems to run 3-4X slower. Both
aren't stunningly fast. The test clients are running on the same machine
as Cassandra, and I'm only getting somewhere between 100-400 (huge
variance) with N::C::Easy and 30-90 with N::C. This test is writing key
value pairs, with the keys being an incrementing numbber, and the values
being a log line from one of our systems (~200 character string). I'm
surprised there is such a huge difference in speed between the two
modules and that the transactions per second are so low even on my
3.2Ghz P4 2GB RAM box. I tried dropping the consistency level down to
zero but it had a negligible affect.
