There's a few things you can do here that might help. First off, if you're using the default heap settings, that's a serious problem. If you've got the head room, my recommendation is to use 16GB heap with 12 GB new gen and pin your memtable heap space to 2GB. Set your max tenuring threshold to 6 and your survivor ratio to 6. You don't need a lot of old gen space with cassandra, almost everything that will show up there is memtable related, and we allocate a *lot* whenever we read data off disk.
Most folks use the default disk read ahead setting of 128KB. You can check this setting using blockdev --report, under the RA column. You'll see 256 there, that's in 512 byte sectors. MVs rely on a read before a write, so for every read off disk you do, you'll pull additional 128KB into your page cache. This is usually a waste and puts WAY too much pressure on your disk. On SSD, I always change this to 4KB. Next, be sure you're setting your compression rate accordingly. I wrote a long post on the topic here: https://thelastpickle.com/blog/2018/08/08/compression_performance.html. Our default compression is very unfriendly for read heavy workloads if you're reading small rows. If your records are small, 4KB compression chunk length is your friend. I have some slides showing pretty good performance improvements from the above 2 changes. Specifically, I went from 16K reads a second at 180ms p99 latency up to 63K reads / second at 21ms p99. Disk usage dropped by a factor of 10. Throw in those JVM changes I recommended and things should improve even further. Generally speaking, I recommend avoiding MVs, as they can be a giant mine if you aren't careful. They're not doing any magic behind the scenes that makes scaling easier, and in a lot of cases they're a hinderance. You still need to understand the underlying data and how it's laid out to use them properly, which is 99% of the work. Jon On Fri, Feb 7, 2020 at 10:32 AM Michael Shuler <mich...@pbandjelly.org> wrote: > That JIRA still says Open, so no, it has not been fixed (unless there's > a fixed duplicate in JIRA somewhere). > > For clarification, you could update that ticket with a comment including > your environmental details, usage of MV, etc. I'll bump the priority up > and include some possible branchX fixvers. > > Michael > > On 2/7/20 10:53 AM, Surbhi Gupta wrote: > > Hi, > > > > We are getting hit by the below bug. > > Other than lowering hinted_handoff_throttle_in_kb to 100 any other work > > around ? > > > > https://issues.apache.org/jira/browse/CASSANDRA-13810 > > > > Any idea if it got fixed in later version. > > We are on Open source Cassandra 3.11.1 . > > > > Thanks > > Surbhi > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org > For additional commands, e-mail: user-h...@cassandra.apache.org > >
