First thing you are going to need is some log messages. From the machine the client was connected to when it returned the UnavailableException.
At CL ANY even Hinted Handoff counts towards meeting the CL for a write http://wiki.apache.org/cassandra/HintedHandoff . So as long as the client can connect to a node in the cluster the write should be accepted. But there could still be reasons why the HH could not be recorded, the logs should help sort it out though. For a read, with RF 2 and 3 nodes down there may be an entire section of the distributed hash table will be unavailable. So any read for a key that is replicated on 2 of the 3 down nodes will fail. I would advise getting those nodes up asap. Keys in all CF's are distributed around the cluster according to the partitioner, replication strategy and initial tokens. If you are using the Random Partitioner, Simple Replication Strategy and set evenly spaced initial tokens then keys will be evenly spaced around the cluster. Hope that helps Aaron On 14 Dec 2010, at 21:12, Rajat Chopra wrote: > Hi! > I have a 16 node cluster with two keyspaces and several column families > within each, all with RF=2. > All the reads/writes work with all column families but only one of them gives > me an unavailable exception, even with CL.ANY consistency. The nodetool ring > shows that three of the nodes of the cluster are down (they have their disk > full because Cassandra runs out of memory and leaves humongous hprof files). > > Anyhow, my question is this – is the unavailable exception being thrown > because that particular column family (with its replica) happened to be > stored in the three nodes that were down? > Is there a way to see what keys/cfs are stored in which node(s) of the > cluster? > > Thanks, > Rajat >
