Write row A, flush into sstable 1 Delete row A, flush the tombstone into sstable 2
The tombstone in sstable 2 can’t be removed until row A in sstable 1 gets removed. If you just keep recompacting sstable 2 by itself, the row in sstable A remains on disk. -- Jeff Jirsa > On Sep 26, 2017, at 2:01 AM, shalom sagges <shalomsag...@gmail.com> wrote: > > Thanks Jeff! > > I'll try that. > I'm not sure I understand how the tombstones are covering data in another > file. Do you have a small example perhaps? > > Thanks again! > >> On Tue, Sep 26, 2017 at 1:38 AM, Jeff Jirsa <jji...@gmail.com> wrote: >> The problem is likely that your sstables overlap - your 91% droppable >> tombstones are probably covering data in another file. Until that data is >> removed, those tombstones cant be dropped. >> >> This is why a full major compaction often works better for simply reclaiming >> disk space (though it takes a lot more disk as it runs and has other >> drawbacks) - it joins the shadowed data with the tombstones into a single >> file. You can sorta fake that by using user-defined compaction with multiple >> sstables to try to compact away the shadowed data - pick 2 or more -Data >> files and compact them together. Works best if you can be sure which data is >> overlapping, but short of that you'll probably want to pick data with >> approximately the same (or older) calendar timestamps. >> >> >> >>> On Mon, Sep 25, 2017 at 11:10 AM, shalom sagges <shalomsag...@gmail.com> >>> wrote: >>> Hi Everyone, >>> >>> I'm running into an issue I can't seem to Solve. >>> >>> I execute force compaction in order to reclaim back storage. >>> Everything was working fine for a time, but after a while I found that >>> tombstones aren't being removed any longer. >>> >>> For example, I've compacted the following SSTable: >>> 21G Sep 19 10:18 file-jb-69103-Data.db >>> Estimated droppable tombstones: 0.9115601492568154 >>> >>> I ran it with jmxterm and the compaction started and completed >>> successfully. >>> $>run -b org.apache.cassandra.db:type=CompactionManager >>> forceUserDefinedCompaction file-jb-69103-Data.db >>> #calling operation forceUserDefinedCompaction of mbean >>> org.apache.cassandra.db:type=CompactionManager >>> #operation returns: >>> null >>> >>> >>> A new SStable was created, but no tombstones were removed. >>> 21G Sep 25 12:16 file-jb-71319-Data.db >>> Estimated droppable tombstones: 0.9115601492568154 >>> >>> I've tried running it from jconsole as well, but the outcome was the same. >>> >>> Is anyone familiar with this issue? >>> Btw, this cluster is on version 2.0.14 . >>> >>> >>> Thanks! >>> Shalom >>> >> >
