> The environment is built using established images for Cassandra 3.10. > Unfortunately the debug log does not indicate any errors before I start > seeing the WARN for missing hints folder. I understand that hints file will > be deleted after replay is complete, but not sure of the root cause of why > the hints folder is getting deleted. > When I look at the nodetool status or nodetool ring - it indicates that > all nodes are up and running in normal state, no node went down. Also, I do > not see anything the debug logs indicating that a node went down. In such a > scenario, I am not sure why would HintsWriterExecutor would get triggered. > > That error code (O_RDONLY) in the log message indicates that the hints folder has had its permission bits set to read only.
We've had several issues with some of the tools doing this type of thing when they are run as the root user. Is this specific node one on which you use any of the tools like sstableloader or similar? If so, are you running them as root? Another thought - if it is on a different partition than the data directory, is there free space left on the underlying device holding: /var/lib/cassandra/hints? -- ----------------- Nate McCall Wellington, NZ @zznate CTO Apache Cassandra Consulting http://www.thelastpickle.com
