Thanks for your reply.
We did a rolling upgrade; one by one, took the node offline, set up the
configuration in the new distribution package and fired it up.
I got that error after I tried to enable row cache, in hope of improving
performance. Tried to update one of the CFs from the cli, but it just hung
and never took effect.
I'm also seeing these exceptions:
ERROR [MutationStage:29] 2011-11-15 14:37:52,213
AbstractCassandraDaemon.java (line 133) Fatal exception in thread
Thread[MutationStage:29,5,main]
java.lang.NullPointerException
at java.util.TreeMap.rotateRight(Unknown Source)
at java.util.TreeMap.fixAfterInsertion(Unknown Source)
at java.util.TreeMap.put(Unknown Source)
at
org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:87)
at
org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:122)
at org.apache.cassandra.db.Memtable.resolve(Memtable.java:218)
at org.apache.cassandra.db.Memtable.put(Memtable.java:134)
at
org.apache.cassandra.db.ColumnFamilyStore.apply(ColumnFamilyStore.java:779)
at org.apache.cassandra.db.Table.apply(Table.java:434)
at org.apache.cassandra.db.RowMutation.apply(RowMutation.java:255)
at
org.apache.cassandra.db.RowMutationVerbHandler.doVerb(RowMutationVerbHandler.java:55)
at
org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:59)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
and
ERROR [CompactionExecutor:8] 2011-11-15 14:28:54,592
AbstractCassandraDaemon.java (line 133) Fatal exception in thread
Thread[CompactionExecutor:8,1,main]
java.lang.AssertionError: Added column does not sort as the last column
at
org.apache.cassandra.db.ArrayBackedSortedColumns.addColumn(ArrayBackedSortedColumns.java:126)
at
org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:122)
at
org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:117)
at
org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:147)
at
org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:231)
at
org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:115)
at
org.apache.cassandra.db.compaction.PrecompactedRow.<init>(PrecompactedRow.java:102)
at
org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:127)
at
org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:102)
--
at
com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
at
com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
at
org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:172)
at
org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:132)
at
org.apache.cassandra.db.compaction.CompactionManager$1.call(CompactionManager.java:114)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
Source)
at java.lang.Thread.run(Unknown Source)
Thanks
silviu
On Tue, Nov 15, 2011 at 12:09 AM, Nate McCall <n...@datastax.com> wrote:
> Can you describe how you did the upgrade on these machines? You may
> still have some old jars on the classpath.
>
> On Mon, Nov 14, 2011 at 4:03 PM, Silviu Matei <silvma...@gmail.com> wrote:
> > Hi
> > I'm getting this error when I try to run describe cluster:
> > [] describe cluster;
> > Cluster Information:
> > Snitch: org.apache.cassandra.locator.SimpleSnitch
> > Partitioner: org.apache.cassandra.dht.RandomPartitioner
> > Schema versions:
> > Error retrieving data: Internal error processing describe_schema_versions
> >
> > with this in the logs:
> > ERROR [pool-2-thread-3] 2011-11-14 21:40:51,975 Cassandra.java (line
> 3540)
> > Internal error processing describe_schema_versions
> > java.lang.NoSuchMethodError:
> >
> org.apache.commons.lang.StringUtils.join(Ljava/util/Collection;Ljava/lang/String;)Ljava/lang/String;
> > at
> >
> org.apache.cassandra.service.StorageProxy.describeSchemaVersions(StorageProxy.java:959)
> > at
> >
> org.apache.cassandra.thrift.CassandraServer.describe_schema_versions(CassandraServer.java:1151)
> > at
> >
> org.apache.cassandra.thrift.Cassandra$Processor$describe_schema_versions.process(Cassandra.java:3536)
> > at
> >
> org.apache.cassandra.thrift.Cassandra$Processor.process(Cassandra.java:2889)
> > at
> >
> org.apache.cassandra.thrift.CustomTThreadPoolServer$WorkerProcess.run(CustomTThreadPoolServer.java:187)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown
> > Source)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> > Source)
> > at java.lang.Thread.run(Unknown Source)
> >
> >
> > Anyone have any insight? It's a 3node 1.0.2 cluster, recently upgraded
> from
> > .87.
> > Thank you
> >
> >
>
