Hi Jared,
Sounds like you have two nodes in the cluster. What is your replication
factor set to? 1? 2?
Have you ever run repair? What consistency level do you use for reads
and writes?
From the way you are speaking it sounds like you are sending all of
your traffic to a single node (primary, secondary).
If you never repair the ranges, and you read the data infrequently (or
only with range slices) I can guess that your data never got replicated
to your secondary node.
Ben
On 3/24/11 9:44 PM, Jared Laprise wrote:
Thanks for the responses. I got everything working again, and have some ideas
on why but am not completely sure.
How I got it working again was simply bring the second node back online. I was
under the assumption that all data is replicated between nodes (eventually). Am
I incorrect? It would seem that each node stores different data and delegates
the read request to whichever node holds the data. Although I've spent a lot of
time with Cassandra in a single node environment I think I may be lacking a bit
of understanding on how Cassandra behaves in a clustered environment.
-----Original Message-----
From: Jonathan Ellis [mailto:jbel...@gmail.com]
Sent: Thursday, March 24, 2011 8:22 PM
To: user@cassandra.apache.org
Cc: aaron morton
Subject: Re: URGENT HELP PLEASE!
Right, Cassandra doesn't keep old versions around so to see an old version you
have to have uncompacted data and whack the new data -- either by blowing away
sstables or not replaying the commitlog.
Snapshots flush before creating their hard links, which rules out any commitlog
problems.
If you ran out of disk space you wouldn't get past the commitlog append, so
you'd never get new data in at all after that.
Sounds like an environmental problem, not Cassandra specific.
On Thu, Mar 24, 2011 at 9:10 PM, aaron morton<aa...@thelastpickle.com> wrote:
Was there anything in the server logs during startup ?
I've not heard of this happening before and it's hard think of how /
why cassandra could revert it's data. Other than something external
playing with the files on disk Aaron On 25 Mar 2011, at 13:49, Jared
Laprise wrote:
Hello all, I'm running 2 Cassandra 6.5 nodes and I brought down the
secondary node and restarted the primary node. After Cassandra came
back up all data has been reverted to several months ago.
I could really use some incite here, this is a production website and
I need to act quickly. I have a cron job that takes a snapshot every
night, but even with that I tried to restore a snapshot on my local
development environment and it was also missing a ton of data.
Any help will be so appreciated.
--
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of DataStax, the source for professional Cassandra support
http://www.datastax.com
--
Ben Coverston
DataStax -- The Apache Cassandra Company
http://www.datastax.com/
