> > Last, but not least: are you using the default number of vnodes, 256? The > overhead of large number of vnodes (times the number of nodes), can be > quite significant. We've seen major improvements in repair runtime after > switching from 256 to 16 vnodes on Cassandra version 3.0.
Is there a recommended procedure to switch the amount of vnodes ? Regards, Leo On Thu, Jun 13, 2019 at 12:06 PM Oleksandr Shulgin < oleksandr.shul...@zalando.de> wrote: > On Thu, Jun 13, 2019 at 10:36 AM R. T. <rastr...@protonmail.com.invalid> > wrote: > >> >> Well, actually by running cfstats I can see that the totaldiskspaceused >> is about ~ 1.2 TB per node in the DC1 and ~ 1 TB per node in DC2. DC2 was >> off for a while thats why there is a difference in space. >> >> I am using Cassandra 3.0.6 and >> my stream_throughput_outbound_megabits_per_sec is th4e default setting so >> according to my version is (200 Mbps or 25 MB/s) >> > > And the other setting: compaction_throughput_mb_per_sec? It is also > highly relevant for repair performance, as streamed in files need to be > compacted with the existing files on the nodes. In our experience change > in compaction throughput limit is almost linearly reflected by the repair > run time. > > The default 16 MB/s is too limiting for any production grade setup, I > believe. We go as high as 90 MB/s on AWS EBS gp2 data volumes. But don't > take it as a gospel, I'd suggest you start increasing the setting (e.g. by > doubling it) and observe how it affects repair performance (and client > latencies). > > Have you tried with "parallel" instead of "DC parallel" mode? The latter > one is really poorly named and it actually means something else, as neatly > highlighted in this SO answer: https://dba.stackexchange.com/a/175028 > > Last, but not least: are you using the default number of vnodes, 256? The > overhead of large number of vnodes (times the number of nodes), can be > quite significant. We've seen major improvements in repair runtime after > switching from 256 to 16 vnodes on Cassandra version 3.0. > > Cheers, > -- > Alex > >
