Awesome explanation :-) Thanks a lot!
On Tue, Sep 26, 2017 at 3:40 PM, Jeff Jirsa <jji...@gmail.com> wrote: > Write row A, flush into sstable 1 > Delete row A, flush the tombstone into sstable 2 > > The tombstone in sstable 2 can’t be removed until row A in sstable 1 gets > removed. If you just keep recompacting sstable 2 by itself, the row in > sstable A remains on disk. > > > > -- > Jeff Jirsa > > > On Sep 26, 2017, at 2:01 AM, shalom sagges <shalomsag...@gmail.com> wrote: > > Thanks Jeff! > > I'll try that. > I'm not sure I understand how the tombstones are covering data in another > file. Do you have a small example perhaps? > > Thanks again! > > On Tue, Sep 26, 2017 at 1:38 AM, Jeff Jirsa <jji...@gmail.com> wrote: > >> The problem is likely that your sstables overlap - your 91% droppable >> tombstones are probably covering data in another file. Until that data is >> removed, those tombstones cant be dropped. >> >> This is why a full major compaction often works better for simply >> reclaiming disk space (though it takes a lot more disk as it runs and has >> other drawbacks) - it joins the shadowed data with the tombstones into a >> single file. You can sorta fake that by using user-defined compaction with >> multiple sstables to try to compact away the shadowed data - pick 2 or more >> -Data files and compact them together. Works best if you can be sure which >> data is overlapping, but short of that you'll probably want to pick data >> with approximately the same (or older) calendar timestamps. >> >> >> >> On Mon, Sep 25, 2017 at 11:10 AM, shalom sagges <shalomsag...@gmail.com> >> wrote: >> >>> Hi Everyone, >>> >>> I'm running into an issue I can't seem to Solve. >>> >>> I execute force compaction in order to reclaim back storage. >>> Everything was working fine for a time, but after a while I found that >>> tombstones aren't being removed any longer. >>> >>> For example, I've compacted the following SSTable: >>> *21G *Sep 19 10:18 file-jb-69103-Data.db >>> Estimated droppable tombstones: >>> >>> *0.9115601492568154* >>> I ran it with jmxterm and the compaction started and completed >>> successfully. >>> $>run -b org.apache.cassandra.db:type=CompactionManager >>> forceUserDefinedCompaction file-jb-69103-Data.db >>> #calling operation forceUserDefinedCompaction of mbean >>> org.apache.cassandra.db:type=CompactionManager >>> #operation returns: >>> null >>> >>> >>> A new SStable was created, but no tombstones were removed. >>> *21G *Sep 25 12:16 file-jb-71319-Data.db >>> Estimated droppable tombstones: >>> >>> * 0.9115601492568154* >>> I've tried running it from jconsole as well, but the outcome was the >>> same. >>> >>> Is anyone familiar with this issue? >>> Btw, this cluster is on version 2.0.14 . >>> >>> >>> Thanks! >>> Shalom >>> >>> >> >
