Thanks, I am using most of the suggested parameters to tune compactions. To clarify, when you say "The sstable_size_in_mb can be thought of a target for the compaction process moving the file beyond L0." do you mean that this property is ignored at memtable flush time, and so memtables are already allowed to be much larger than sstable_size_in_mb?
On Tue, Oct 27, 2015 at 2:57 PM, Nate McCall <n...@thelastpickle.com> wrote: > The sstable_size_in_mb can be thought of a target for the compaction > process moving the file beyond L0. > > Note: If there are more than 32 SSTables in L0, it will switch over to > doing STCS for L0 (you can disable this behavior by passing > -Dcassandra.disable_stcs_in_l0=true as a system property). > > With a lot of overwrites, the settings you want to tune will be > gc_grace_seconds in combination with tombstone_threhsold, > tombstone_compaction_interval and maybe unchecked_tombstone_compaction > (there are different opinions about this last one, YMMV). Making these more > aggressive and increasing your sstable_size_in_mb will allow for > potentially capturing more overwrites in a level which will lead to less > fragmentation. However, making the size too large will keep compaction from > triggering on further out levels which can then exacerbate problems > particulary if you have long-lived TTLs. > > In general, it is very workload specific, but monitoring the histogram for > the number of ssables used in a read (via > org.apache.cassandra.metrics.ColumnFamily.$KEYSPACE.$TABLE.SSTablesPerReadHistogram.95percentile > or shown manually in nodetool cfhistograms output) after any change will > help you narrow in a good setting. > > See > http://docs.datastax.com/en/cql/3.1/cql/cql_reference/compactSubprop.html?scroll=compactSubprop__compactionSubpropertiesLCS > for more details. > > On Tue, Oct 27, 2015 at 3:42 PM, Dan Kinder <dkin...@turnitin.com> wrote: > > > > Hi all, > > > > The docs indicate that memtables are triggered to flush when data in the > commitlog is expiring or based on memtable_flush_period_in_ms. > > > > But LCS has a specified sstable size; when using LCS are memtables > flushed when they hit the desired sstable size (default 160MB) or could L0 > sstables be much larger than that? > > > > Wondering because I have an overwrite workload where larger memtables > would be helpful, and if I need to increase my LCS sstable size in order to > allow for that. > > > > -dan > > > > > -- > ----------------- > Nate McCall > Austin, TX > @zznate > > Co-Founder & Sr. Technical Consultant > Apache Cassandra Consulting > http://www.thelastpickle.com > -- Dan Kinder Senior Software Engineer Turnitin – www.turnitin.com dkin...@turnitin.com
