Hi, I have a cluster of 3 nodes using Cassandra 3.11.1 running on AWS using EBS disks.
I am having some issue with a repair of a keyspace with the command 'nodetool repair my_ks' Validation failed in /10.1.20.10 (progress: 0%) [2018-08-31 15:01:30,566] Some repair failed [2018-08-31 15:01:30,569] Repair command #1 finished in 4 seconds error: Repair job has failed with the error message: [2018-08-31 15:01:30,566] Some repair failed -- StackTrace -- java.lang.RuntimeException: Repair job has failed with the error message: [2018-08-31 15:01:30,566] Some repair failed at org.apache.cassandra.tools.RepairRunner.progress(RepairRunner.java:116) at org.apache.cassandra.utils.progress.jmx.JMXNotificationProgressListener.handleNotification(JMXNotificationProgressListener.java:77) at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.dispatchNotification(ClientNotifForwarder.java:583) at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.doRun(ClientNotifForwarder.java:533) at com.sun.jmx.remote.internal.ClientNotifForwarder$NotifFetcher.run(ClientNotifForwarder.java:452) at com.sun.jmx.remote.internal.ClientNotifForwarder$LinearExecutor$1.run(ClientNotifForwarder.java:108) This keyspace contain few tables and If try to run a repair of single tables using 'nodetool repair my_ks my_table' all seems ok and they repair end successfully. Using the 'scrub' sub-command of nodetool should fix this. When I run it I get some warning but it doesn't fail so I think it does its task. Right? root@ip-10-1-21-10:~# nodetool scrub my_ks WARN 15:22:48,040 Small commitlog volume detected at /var/lib/cassandra/commitlog; setting commitlog_total_space_in_mb to 254. You can override this in cassandra.yaml WARN 15:22:48,047 Small cdc volume detected at /var/lib/cassandra/cdc_raw; setting cdc_total_space_in_mb to 984. You can override this in cassandra.yaml WARN 15:22:48,048 Only 1.226GiB free across all data volumes. Consider adding more capacity to your cluster or removing obsolete snapshots root@ip-10-1-21-10:~# echo $? 0 Anyway this doesn't fix the my issue. Any suggestions? Regards, Marco
