Hi guys,
I am running cassandra 3.0.5 on 5 nodes, NetworkTopologyStrategy, factor 2
I had to add new 3 columns to an existing type and I've done it through
cqlsh.
After one of the alter commands I've got
*OperationTimedOut: errors={'x.x.x.x': 'Request timed out while waiting for
schema agreement. See Session.execute[_async](timeout) and
Cluster.max_schema_agreement_wait.'}, last_host=x.x.x.x*
I've checked with describe schema and the new fields were in place, so I
suppose that cql tool timed out.
After this I've noticed that the application (connecting through datastax
java driver, quorum consistency level) started to throw
*com.datastax.driver.core.exceptions.ReadFailureException: Cassandra
failure during read query at consistency QUORUM (2 responses were required
but only 0 replica responded, 1 failed)*
I thought that this might be related to the schema changed and that repair
would force schema propagation so I run nodetool repair on the box on which
I trigged the schema alteration.
Then I got
*... Validation failed in /x.x.x.y (progress: 0%)*
on this node and
*ERROR [CompactionExecutor:37434] 2016-12-06 14:03:38,543
CassandraDaemon.java:195 - Exception in thread
Thread[CompactionExecutor:37434,1,main]*
*java.lang.AssertionError: null*
* at
org.apache.cassandra.db.rows.ComplexColumnData$Builder.addCell(ComplexColumnData.java:246)
~[apache-cassandra-3.0.5.jar:3.0.5]*
* at
org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:613)
~[apache-cassandra-3.0.5.jar:3.0.5]*
* at
org.apache.cassandra.db.rows.Row$Merger$ColumnDataReducer.getReduced(Row.java:539)
~[apache-cassandra-3.0.5.jar:3.0.5]*
* at
org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:220)
~[apache-cassandra-3.0.5.jar:3.0.5]*
* at
org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:159)
~[apache-cassandra-3.0.5.jar:3.0.5]*
* at
org.apache.cassandra.utils.AbstractIterator.hasNext(AbstractIterator.java:47)
~[apache-cassandra-3.0.5.jar:3.0.5]*
* at org.apache.cassandra.db.rows.Row$Merger.merge(Row.java:516)
~[apache-cassandra-3.0.5.jar:3.0.5]*
*....*
* at
org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionCandidate.run(CompactionManager.java:263)
~[apache-cassandra-3.0.5.jar:3.0.5]*
* at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
~[na:1.8.0_60]*
* at java.util.concurrent.FutureTask.run(FutureTask.java:266)
~[na:1.8.0_60]*
* at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
~[na:1.8.0_60]*
* at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_60]*
* at java.lang.Thread.run(Thread.java:745) [na:1.8.0_60]*
On the node /x.x.x.y
Do you any suggestion?
Thank you in advance,
Robert
