Hi Bowen,
Thanks for the reply, these have been our normal shutdowns, so we do a nodetool
drain before restarting the service, so I would have thought there should not
be any commtlogs
However there is these messages for one commit log, But looks like it has
finished quickly and correctly:
IN
Nothing obvious from the logs you posted.
Generally speaking, replaying commit log is often the culprit when a
node takes a long time to start. I have seen many nodes with large
memtable and commit log size limit spending over half an hour replaying
the commit log. I usually do a "nodetool flu
Hi all,
We have upgraded a couple of clusters from 3.11.6, now we are having issues
when we restart the nodes.
The node will either hang or take 10-30 minute to restart, these are the last
messages we have in the system.log:
INFO [NonPeriodicTasks:1] 2022-01-19 10:08:23,267 FileUtils.java:54
We use nodetool repair -pr -full. We have scheduled these to run
automatically. For us also it has been seamless on most of the clusters.
This particular node is misbehaving for reasons unknown to me. As per your
suggestion, going through system.logs to find that unknown. Will keep you
posted if am
May I ask how do you run the repair? Is it manually via the nodetool
command line tool, or a tool or script, such as Cassandra Reaper? If you
are running the repairs manually, would you mind give Cassandra Reaper a
try?
I have a fairly large cluster under my management, and last time I tried
