Hi, thanks for sharing your tests ! Though, how did you inserted the data ? Did you try adding columns in an atomic and random order, with a small memtable size to achieve a big sharding (normal in time series use case) ?
I think performing ./md_test against this set of data would be interesting, taking care to shard enough to have parts of each key on each SSTable, or in many at least. This would measure the effectiveness of this parameter in a "normal" time series workflow (which is a standard use case of C*). By the way, this might be done on both LCS <http://www.datastax.com/dev/blog/leveled-compaction-in-apache-cassandra> and STCS <http://www.datastax.com/documentation/cql/3.1/cql/cql_reference/tabProp.html?scroll=tabProp__moreCompaction> and see how both behave. Thanks again, it is interesting to hear about tests like yours ! C*heers Alain 2015-01-30 17:32 GMT+01:00 Ravi Agrawal <ragra...@clearpoolgroup.com>: > I did a small test. I wrote data to 4 different column family. 30MB of > data. > > 256 rowkeys and 100K columns on an average. > > And then deleted all data from all of them. > > > > 1. Md_normal - created using default compaction parameters and Gc > Grace seconds was 5 seconds. Data was written and then deleted. Compaction > was ran using "nodetool compact keyspace columnfamily" - I see full disk > data, but cannot query columns(since data was deleted consistent behavior) > and cannot query rows in cqlsh. Hits timeout. > > 2. Md_test - created using following compact parameters - > "compaction={'tombstone_threshold': '0.000001', 'class': > 'SizeTieredCompactionStrategy'}" and Gc Grace seconds was 5 seconds. > Disksize is reduced, and am able to query rows which return 0. > > 3. Md_test2 - created using following compact parameters - > "compaction={'tombstone_threshold': '0.0', 'class': > 'SizeTieredCompactionStrategy'}". Disksize is reduced, not able to query > rows using cqlsh. Hits timeout. > > 4. Md_forcecompact - created using compaction parameters > "compaction={'unchecked_tombstone_compaction': 'true', 'class': > 'SizeTieredCompactionStrategy'}" and Gc Grace seconds was 5 seconds. Data > was written and then deleted. I see full disk data, but cannot query any > data using mddbreader and cannot query rows in cqlsh. Hits timeout. > > > > Next day sizes were - > > 30M ./md_forcecompact > > 4.0K ./md_test > > 304K ./md_test2 > > 30M ./md_normal > > > > Feel of the data that we have is - > > 8000 rowkeys per day and columns are added throughout the day. 300K > columns on an average per rowKey. > > > > > > > > *From:* Alain RODRIGUEZ [mailto:arodr...@gmail.com] > *Sent:* Friday, January 30, 2015 4:26 AM > > *To:* user@cassandra.apache.org > *Subject:* Re: Tombstone gc after gc grace seconds > > > > The point is that all the "parts" or "fragments" of the row need to be in > the SSTables implied in the compaction for C* to be able to evict the row > effectively. > > > > My understanding of those parameters is that they will trigger a > compaction on the SSTable that exceed this ratio. This will work properly > if you never "update" a row (by modifying a value or adding a column). If > your workflow is something like "Write once per partition key", this > parameter will do the job. > > > > If you have fragments, you might trigger this compaction for nothing. In > the case of frequently updated rows (like when using wide rows / time > series) your only way to get rid of tombstone is a major compaction. > > > > That's how I understand this. > > > > Hope this help, > > > > C*heers, > > > > Alain > > > > 2015-01-30 1:29 GMT+01:00 Mohammed Guller <moham...@glassbeam.com>: > > Ravi - > > > > It may help. > > > > What version are you running? Do you know if minor compaction is getting > triggered at all? One way to check would be see how many sstables the data > directory has. > > > > Mohammed > > > > *From:* Ravi Agrawal [mailto:ragra...@clearpoolgroup.com] > *Sent:* Thursday, January 29, 2015 1:29 PM > *To:* user@cassandra.apache.org > *Subject:* RE: Tombstone gc after gc grace seconds > > > > Hi, > > I saw there are 2 more interesting parameters - > > a. tombstone_threshold - A ratio of garbage-collectable tombstones > to all contained columns, which if exceeded by the SSTable triggers > compaction (with no other SSTables) for the purpose of purging the > tombstones. Default value - 0.2 > > b. unchecked_tombstone_compaction - True enables more aggressive > than normal tombstone compactions. A single SSTable tombstone compaction > runs without checking the likelihood of success. Cassandra 2.0.9 and later. > > Could I use these to get what I want? > > Problem I am encountering is even long after gc_grace_seconds I see no > reduction in disk space until I run compaction manually. I was thinking to > make tombstone threshold close to 0 and unchecked compaction set to true. > > Also we are not running nodetool repair on weekly basis as of now. > > > > *From:* Eric Stevens [mailto:migh...@gmail.com <migh...@gmail.com>] > *Sent:* Monday, January 26, 2015 12:11 PM > *To:* user@cassandra.apache.org > *Subject:* Re: Tombstone gc after gc grace seconds > > > > My understanding is consistent with Alain's, there's no way to force a > tombstone-only compaction, your only option is major compaction. If you're > using size tiered, that comes with its own drawbacks. > > > > I wonder if there's a technical limitation that prevents introducing a > shadowed data cleanup style operation (overwritten data, including deletes, > plus tombstones past their gc grace period); or maybe even couple it > directly with cleanup since most of the work (rewriting old SSTables) would > be identical. I can't think of something off the top of my head, but it > would be so useful that it seems like there's got to be something I'm > missing. > > > > On Mon, Jan 26, 2015 at 4:15 AM, Alain RODRIGUEZ <arodr...@gmail.com> > wrote: > > I don't think that such a thing exists as SSTables are immutable. You > compact it entirely or you don't. Minor compaction will eventually evict > tombstones. If it is too slow, AFAIK, the "better" solution is a major > compaction. > > > > C*heers, > > > > Alain > > > > 2015-01-23 0:00 GMT+01:00 Ravi Agrawal <ragra...@clearpoolgroup.com>: > > Hi, > > I want to trigger just tombstone compaction after gc grace seconds is > completed not nodetool compact keyspace column family. > > Anyway I can do that? > > > > Thanks > > > > > > > > > > >
