tl;dr: Decommissioning datacenters by running nodetool decommission on a node deletes the data on the decommissioned node - is this expected ?
I am trying our some tests on my multi-datacenter setup. Somewhere in the docs I read that decommissioning a node will stream its data to other nodes but it still retains its copy of the data. I was expecting the same behavior with multiple datacenters. I am using cassandra 1.2.12. Following are my observations: Lets say I have a datacenter DC1 which has keyspace "keyspace_dc_1" and I have another datacenter DC2 which has keyspace "keyspace_dc_2". They already have some data in them. I add DC2 to DC1, update the replication factors on both the keyspaces. Looking at the gossipinfo, I can see that the schemas are synced. I then look at the cfstats output and I can see then both the keyspaces are replicated on both the datacenters (also on the disk, as I can see a non-zero sstable count). Now, I decommission DC2: 1) Update the replication factors for the keyspaces. 2) Run nodetool decommission on all the nodes. I see that I have lost all my keyspaces (and data), the keyspaces from DC1 and DC2. This does not seem normal to me, is this expected ? Thanks, Sandeep
