Hi Guys,
Im testing out Cassandra with a 2.0.1 install, just like with 1.2.8 I last
tested, from time to time I am coming across errors that prevent Cassandra
starting up.
For example: Yesterday I installed this instance, created and removed some
“tables” using cql3 and inserted/updated/truncated some tables. Overnight, not
sure why, cassandra stopped running and won’t start up again:
INFO 10:34:37,776 Finished reading
/cassandra/commitlog/CommitLog-3-1382935041799.log
INFO 10:34:37,776 Replaying /cassandra/commitlog/CommitLog-3-1383089534960.log
INFO 10:34:37,796 Finished reading
/cassandra/commitlog/CommitLog-3-1383089534960.log
INFO 10:34:37,796 Skipped 11 mutations from unknown (probably removed) CF with
id 941b25fd-703f-3013-aadd-40b18cbcd9cd
ERROR 10:34:37,797 Exception encountered during startup
java.lang.RuntimeException: java.util.concurrent.ExecutionException:
java.lang.IndexOutOfBoundsException: index (2) must be less than size (2)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:411)
at
org.apache.cassandra.utils.FBUtilities.waitOnFutures(FBUtilities.java:400)
at
org.apache.cassandra.db.commitlog.CommitLogReplayer.blockForWrites(CommitLogReplayer.java:105)
at
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:147)
at
org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:126)
at
org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:299)
at
org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:442)
at
org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:485)
Caused by: java.util.concurrent.ExecutionException:
java.lang.IndexOutOfBoundsException: index (2) must be less than size (2)
at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
at java.util.concurrent.FutureTask.get(FutureTask.java:111)
at
org.apache.cassandra.utils.FBUtilities.waitOnFuture(FBUtilities.java:407)
... 7 more
Normally I just delete the files and restart the node. If you are in a
production environment, how do you deal with this type of issue?
(I suspect there may be some sort of bug with replaying commit logs in 2.0.1,
probably associated with replaying a commit log after you have deleted a table,
and recreated with a different definition.)
Best regards,
Jacob
