C* 1.2.3 , three node cluster. Yesterday we stumbled upon this error, and it's the first time we ever see it:
ERROR [ReadStage:13918] 2013-04-08 21:09:54,714 CassandraDaemon.java (line 164) Exception in thread Thread[ReadStage:13918,5,main] java.lang.RuntimeException: java.lang.IllegalArgumentException at org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:64) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:679) Caused by: java.lang.IllegalArgumentException at java.nio.Buffer.limit(Buffer.java:266) at org.apache.cassandra.db.marshal.AbstractCompositeType.getBytes(AbstractCompositeType.java:51) at org.apache.cassandra.db.marshal.AbstractCompositeType.getWithShortLength(AbstractCompositeType.java:60) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:78) at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31) at java.util.TreeMap.getEntryUsingComparator(TreeMap.java:368) at java.util.TreeMap.getEntry(TreeMap.java:339) at java.util.TreeMap.get(TreeMap.java:272) at org.apache.cassandra.db.TreeMapBackedSortedColumns.getColumn(TreeMapBackedSortedColumns.java:175) at org.apache.cassandra.db.AbstractColumnContainer.getColumn(AbstractColumnContainer.java:114) at org.apache.cassandra.db.filter.ExtendedFilter$FilterWithClauses.isSatisfiedBy(ExtendedFilter.java:284) at org.apache.cassandra.db.ColumnFamilyStore.filter(ColumnFamilyStore.java:1490) at org.apache.cassandra.db.ColumnFamilyStore.getRangeSlice(ColumnFamilyStore.java:1449) at org.apache.cassandra.service.RangeSliceVerbHandler.executeLocally(RangeSliceVerbHandler.java:46) at org.apache.cassandra.service.RangeSliceVerbHandler.doVerb(RangeSliceVerbHandler.java:58) ... 4 more I see that there is a similar past bug (https://issues.apache.org/jira/browse/CASSANDRA-5186) solved in 1.1 about concurrent CF creation. The problem here is that there was no CF creation going on, the only *possible* cause was a bogus CQL select that was executing selectiong over a field that didn't have an index, but I was not able to reproduce it systematically. -- Marco Matarazzo