+1 to checking for snapshots. Cassandra by default will automatically snapshot tables before destructive actions like drop or truncate.
Some general advice regarding cleanup. Cleanup will result in a temporary increase in both disk I/O load and disk space usage (especially with STCS). It should only be used as part of a planned increase in capacity when you still have plenty of disk space left on your existing nodes. If you are running Cassandra in the cloud (AWS, Azure etc) you can add an EBS volume, copy your sstables to it then bind mount it to the troubled CF directory. This will give you some emergency disk space to let compaction and cleanup do its thing safely. On Tue, 16 Feb 2016 at 10:57 Robert Coli <rc...@eventbrite.com> wrote: > On Sat, Feb 13, 2016 at 4:30 PM, Branton Davis <branton.da...@spanning.com > > wrote: > >> We use SizeTieredCompaction. The nodes were about 67% full and we were >> planning on adding new nodes (doubling the cluster to 6) soon. >> > > Be sure to add those new nodes one at a time. > > Have you checked for, and cleared, old snapshots? Snapshots are > automatically taken at various times and have the unusual property of > growing larger over time. This is because they are hard links of data files > and do not take up disk space of their own until the files they link to are > compacted into new files. > > =Rob > > -- Ben Bromhead CTO | Instaclustr <https://www.instaclustr.com/> +1 650 284 9692 Managed Cassandra / Spark on AWS, Azure and Softlayer
