Re: unlimited repair throughput in Cassandra 4.0

2022-06-14 Thread Azamat Hackimov
Hello. I had the same issues on full repair. I've checked on various GC settings, the most performant is ZGC on Java 11, but I had some stability issues. I left G1GC settings from 3.11.x and got the same issues as yours: CPU load over 90 %, and growing count of open file descriptors (up to max all

Re: MUTATION_* messages in log

2022-04-19 Thread Azamat Hackimov
ogs, and looking for long > GC pauses. > > > On 12/04/2022 10:15, Azamat Hackimov wrote: > > Hello. > > > > I've got two DCs with 8 hardware servers on each. After migration to > > Cassandra 4 I have strange messages in system.log: > > >

MUTATION_* messages in log

2022-04-12 Thread Azamat Hackimov
Hello. I've got two DCs with 8 hardware servers on each. After migration to Cassandra 4 I have strange messages in system.log: INFO [ScheduledTasks:1] 2022-04-12 12:05:58,595 MessagingMetrics.java:206 - MUTATION_RSP messages were dropped in last 5000 ms: 0 internal and 241 cross node. Mean inter

Re: Cassandra 4.0 randomly freezes on heavy load

2022-02-25 Thread Azamat Hackimov
OK, I think I figured out what was wrong in my setup. Appears that during the upgrade course I totally missed Garbage Collector settings in jvm11-server.options. After installation from RPM it defines the default CMS garbage collector, which is totally wrong for my hi-load environment. I returned b

Re: Cassandra 4.0 randomly freezes on heavy load

2022-02-25 Thread Azamat Hackimov
gt; when the network based filesystem loses their connection and reconnects. > > On 25/02/2022 06:09, Azamat Hackimov wrote: > > Hello! > > > > I recently migrated Cassandra from 3.11.x to 4.0 and got strange > > freezes on heavy load. It looks like some nodes in DC stopp

Cassandra 4.0 randomly freezes on heavy load

2022-02-24 Thread Azamat Hackimov
Hello! I recently migrated Cassandra from 3.11.x to 4.0 and got strange freezes on heavy load. It looks like some nodes in DC stopped responding and got DN status. I cannot check status directly on node via nodetool status or even restart Cassandra with `systemctl restart cassandra` command. Only