Hi,


it's not only performance issue. In case of network problem writer tread can be 
blocked, also in case of failure loss of data can occur.



Best regards, Vladimir Yudovin, 

Winguzone - Hosted Cloud Cassandra
Launch your cluster in minutes.





---- On Tue, 01 Nov 2016 14:10:10 -0400John Sanda <john.sa...@gmail.com> 
wrote ----




I know that using NFS is discouraged, particularly for the commit log. Can 
anyone shed some light into what kinds of problems I might encounter aside from 
performance? The reason for my inquiry is because I have some deployments with 
Cassandra 2.2.1 that use NFS and are experiencing some problems like 
reoccurring corrupted commit log segments on start up:



 ERROR 19:38:42 Exiting due to error while processing commit log during 
initialization. 
org.apache.cassandra.db.commitlog.CommitLogReplayer$CommitLogReplayException: 
Mutation checksum failure at 33296351 in CommitLog-5-1474325237114.log at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.handleReplayError(CommitLogReplayer.java:622)
 [apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.replaySyncSection(CommitLogReplayer.java:492)
 [apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:388)
 [apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:147)
 [apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:189) 
[apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:169) 
[apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:266) 
[apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:488) 
[apache-cassandra-2.2.1.jar] at 
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:595) 
[apache-cassandra-2.2.1.jar]



In one deployment after removing all of corrupted commit log segments I got a 
different error:



Exception (java.lang.RuntimeException) encountered during startup: 
java.nio.file.NoSuchFileException: 
/cassandra_data/data/hawkular_metrics/metrics_idx-1ea881506ee311e6890b131c1bc89929/la-86-big-Statistics.db
 java.lang.RuntimeException: java.nio.file.NoSuchFileException: 
/cassandra_data/data/hawkular_metrics/metrics_idx-1ea881506ee311e6890b131c1bc89929/la-86-big-Statistics.db
 at org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:55) 
at org.apache.cassandra.io.util.ChannelProxy.<init>(ChannelProxy.java:66) 
at 
org.apache.cassandra.io.util.RandomAccessReader.open(RandomAccessReader.java:78)
 at 
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:91)
 at 
org.apache.cassandra.io.sstable.metadata.MetadataSerializer.deserialize(MetadataSerializer.java:101)
 at 
org.apache.cassandra.db.ColumnFamilyStore.removeUnfinishedCompactionLeftovers(ColumnFamilyStore.java:672)
 at 
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:216) at 
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:488) 
at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:595) 
Caused by: java.nio.file.NoSuchFileException: 
/cassandra_data/data/hawkular_metrics/metrics_idx-1ea881506ee311e6890b131c1bc89929/la-86-big-Statistics.db
 at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86) at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102) at 
sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107) at 
sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:177)
 at java.nio.channels.FileChannel.open(FileChannel.java:287) at 
java.nio.channels.FileChannel.open(FileChannel.java:335) at 
org.apache.cassandra.io.util.ChannelProxy.openChannel(ChannelProxy.java:51) ... 
8 more



The latter error looks like it involves compaction and might be unrelated. I 
don't know if it matters, but I have commit log compression enabled in these 
environments.



-- 



- John









Reply via email to