Hi!
We’re running Cassandra 3.9
On the application side I see failed reads with this exception
com.datastax.driver.core.exceptions.ReadFailureException: Cassandra failure
during read query at consistency QUORUM (2 responses were required but only 0
replica responded, 2 failed)
On the server side we see:
WARN [SharedPool-Worker-3] 2016-10-28 13:28:22,965
AbstractLocalAwareExecutorService.java:169 - Uncaught exception on thread
Thread[SharedPool-Worker-3,5,
main]: {}
java.lang.AssertionError: null
at org.apache.cassandra.db.rows.BTreeRow.getCell(BTreeRow.java:212)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.db.SinglePartitionReadCommand.canRemoveRow(SinglePartitionReadCommand.java:899)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.db.SinglePartitionReadCommand.reduceFilter(SinglePartitionReadCommand.java:863)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndSSTablesInTimestampOrder(SinglePartitionReadCommand.java:748)
~[apache-cassan
dra-3.7.jar:3.7]
at
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDiskInternal(SinglePartitionReadCommand.java:519)
~[apache-cassandra-3.7.jar:
3.7]
at
org.apache.cassandra.db.SinglePartitionReadCommand.queryMemtableAndDisk(SinglePartitionReadCommand.java:496)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.db.SinglePartitionReadCommand.queryStorage(SinglePartitionReadCommand.java:358)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.db.ReadCommand.executeLocally(ReadCommand.java:366)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.db.ReadCommandVerbHandler.doVerb(ReadCommandVerbHandler.java:48)
~[apache-cassandra-3.7.jar:3.7]
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64)
~[apache-cassandra-3.7.jar:3.7]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_102]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164)
~[apache-cassandra-
3.7.jar:3.7]
at
org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$LocalSessionFutureTask.run(AbstractLocalAwareExecutorService.java:136)
[apache
-cassandra-3.7.jar:3.7]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105)
[apache-cassandra-3.7.jar:3.7]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_102]
It’s only affect single table. Sadly both on test (3.9) and production (3.7)
deployments of cassandra.
What could be the problem? Please help.
