I haven't throttled cassandra compactions. My assumption is that because we are running lot of batch jobs which can updates or delete partition key, number of compactions are too many to finish in time which i see in nodetool compactionstats pending tasks.
Multiple Batch write (and delete) of partition keys. Each batch can have overlap of partition keys. High throughput read after couple of hours multiple batch writes On Fri, Sep 15, 2017 at 9:07 PM, Jeff Jirsa <jji...@gmail.com> wrote: > Do you have compaction throttled now? > What exactly takes 6-8 hours? > Are you writing all the keys at one time, then reading, then deleting them > all, or is it a constant stream of writes and reads? > > > -- > Jeff Jirsa > > > > On Sep 15, 2017, at 8:40 PM, Ashish Pandey <apandey...@gmail.com> wrote: > > > > Hi All, > > > > We are using cassandra 3.9 and experiencing read latency issues. > > > > We are using LeveledCompactionStrategy after experiencing read latency > > issue with SizeTieredCompactionStrategy. For our use case, table keys are > > getting batch updated and batch delete of column values for keys. We > almost > > write to entire set of keys everyday. > > After switching to leveledcompactionstrategy read latency for the table > > has improved but we see that compaction takes lot of time (6-8hrs) and if > > the reads have to happen before compactions are reasonably done, latency > is > > severly impacted. Is there way to increase the speed of compactions as > the > > system resources are available in our use case for compaction to run > faster > > after batch update and delete are run? > > > > This article > > https://www.datastax.com/dev/blog/cassandra-anti-patterns- > queues-and-queue-like-datasets > > suggests > > this is an anti-pattern for cassandra for such use cases, but I would > like > > to try if increase in compaction speed could help. > > > > Thanks very much, appreciate responses. > > > > Ashish > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org > For additional commands, e-mail: dev-h...@cassandra.apache.org > >
