I did a small test. I wrote data to 4 different column family. 30MB of data.
256 rowkeys and 100K columns on an average.
And then deleted all data from all of them.
1. Md_normal - created using default compaction parameters and Gc Grace
seconds was 5 seconds. Data was written and then deleted. Compaction was ran
using "nodetool compact keyspace columnfamily" - I see full disk data, but
cannot query columns(since data was deleted consistent behavior) and cannot
query rows in cqlsh. Hits timeout.
2. Md_test - created using following compact parameters -
"compaction={'tombstone_threshold': '0.000001', 'class':
'SizeTieredCompactionStrategy'}" and Gc Grace seconds was 5 seconds. Disksize
is reduced, and am able to query rows which return 0.
3. Md_test2 - created using following compact parameters -
"compaction={'tombstone_threshold': '0.0', 'class':
'SizeTieredCompactionStrategy'}". Disksize is reduced, not able to query rows
using cqlsh. Hits timeout.
4. Md_forcecompact - created using compaction parameters
"compaction={'unchecked_tombstone_compaction': 'true', 'class':
'SizeTieredCompactionStrategy'}" and Gc Grace seconds was 5 seconds. Data was
written and then deleted. I see full disk data, but cannot query any data using
mddbreader and cannot query rows in cqlsh. Hits timeout.
Next day sizes were -
30M ./md_forcecompact
4.0K ./md_test
304K ./md_test2
30M ./md_normal
Feel of the data that we have is -
8000 rowkeys per day and columns are added throughout the day. 300K columns on
an average per rowKey.
From: Alain RODRIGUEZ [mailto:arodr...@gmail.com]
Sent: Friday, January 30, 2015 4:26 AM
To: user@cassandra.apache.org
Subject: Re: Tombstone gc after gc grace seconds
The point is that all the "parts" or "fragments" of the row need to be in the
SSTables implied in the compaction for C* to be able to evict the row
effectively.
My understanding of those parameters is that they will trigger a compaction on
the SSTable that exceed this ratio. This will work properly if you never
"update" a row (by modifying a value or adding a column). If your workflow is
something like "Write once per partition key", this parameter will do the job.
If you have fragments, you might trigger this compaction for nothing. In the
case of frequently updated rows (like when using wide rows / time series) your
only way to get rid of tombstone is a major compaction.
That's how I understand this.
Hope this help,
C*heers,
Alain
2015-01-30 1:29 GMT+01:00 Mohammed Guller
<moham...@glassbeam.com<mailto:moham...@glassbeam.com>>:
Ravi -
It may help.
What version are you running? Do you know if minor compaction is getting
triggered at all? One way to check would be see how many sstables the data
directory has.
Mohammed
From: Ravi Agrawal
[mailto:ragra...@clearpoolgroup.com<mailto:ragra...@clearpoolgroup.com>]
Sent: Thursday, January 29, 2015 1:29 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: RE: Tombstone gc after gc grace seconds
Hi,
I saw there are 2 more interesting parameters -
a. tombstone_threshold - A ratio of garbage-collectable tombstones to all
contained columns, which if exceeded by the SSTable triggers compaction (with
no other SSTables) for the purpose of purging the tombstones. Default value -
0.2
b. unchecked_tombstone_compaction - True enables more aggressive than
normal tombstone compactions. A single SSTable tombstone compaction runs
without checking the likelihood of success. Cassandra 2.0.9 and later.
Could I use these to get what I want?
Problem I am encountering is even long after gc_grace_seconds I see no
reduction in disk space until I run compaction manually. I was thinking to make
tombstone threshold close to 0 and unchecked compaction set to true.
Also we are not running nodetool repair on weekly basis as of now.
From: Eric Stevens [mailto:migh...@gmail.com]
Sent: Monday, January 26, 2015 12:11 PM
To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
Subject: Re: Tombstone gc after gc grace seconds
My understanding is consistent with Alain's, there's no way to force a
tombstone-only compaction, your only option is major compaction. If you're
using size tiered, that comes with its own drawbacks.
I wonder if there's a technical limitation that prevents introducing a shadowed
data cleanup style operation (overwritten data, including deletes, plus
tombstones past their gc grace period); or maybe even couple it directly with
cleanup since most of the work (rewriting old SSTables) would be identical. I
can't think of something off the top of my head, but it would be so useful that
it seems like there's got to be something I'm missing.
On Mon, Jan 26, 2015 at 4:15 AM, Alain RODRIGUEZ
<arodr...@gmail.com<mailto:arodr...@gmail.com>> wrote:
I don't think that such a thing exists as SSTables are immutable. You compact
it entirely or you don't. Minor compaction will eventually evict tombstones. If
it is too slow, AFAIK, the "better" solution is a major compaction.
C*heers,
Alain
2015-01-23 0:00 GMT+01:00 Ravi Agrawal
<ragra...@clearpoolgroup.com<mailto:ragra...@clearpoolgroup.com>>:
Hi,
I want to trigger just tombstone compaction after gc grace seconds is completed
not nodetool compact keyspace column family.
Anyway I can do that?
Thanks
