Hi,
Cassandra's documentation has several recommendations for moving the commit
log directory to a dedicated disk, separated from the sstables disk(s).
However, I couldn't find much information on what would be good practices
regarding this dedicated commit log disk and I'm wondering how to configure
it the best?
We have a Cassandra cluster running 3.11.1 with separate disks for
sstables/commit logs which worked great until a few weeks, but since a
couple of weeks, some nodes sometimes stop working due to a disk full error:
* The error is: "org.apache.cassandra.io.FSWriteError:
java.nio.file.FileSystemException:
/var/lib/cassandra/commitlog/CommitLog-6-1532888559853.log: No space left
on device"
* we have a dedicated disk of 20 GB
* our commit-related settings are as follow (mostly default):
commit_failure_policy: stop
commitlog_directory: /var/lib/cassandra/commitlog
commitlog_segment_size_in_mb: 32
commitlog_sync: periodic
commitlog_sync_period_in_ms: 10000
commitlog_total_space_in_mb: 19000
We only increased commitlog_total_space_in_mb so that Cassandra fully uses
the dedicated disk, but that may be an error?
The default value for this setting is (per the documentation):
The default value is the smaller of 8192, and 1/4 of the total space of
the commitlog volume.
But that doesn't say much (or should it really by 25% of the disk space?)
So, my questions would be:
* What size should I dedicate to this commit log disk? What are the rules
of thumb to discover the "best" size?
* How should I configure the "commitlog_total_space_in_mb" setting
respectively to the size of the disk?
Thanks!
Jonathan
