If you run sstablekeys it will tell you which key is out of orde --
assuming your index and data files contain the same keys, which if you
have a corrupt sstable may or may not be the case.

On Wed, Jan 19, 2011 at 1:10 AM, David King <dk...@ketralnis.com> wrote:
> I'm upgrading an 0.6 cluster to 0.7 in a testing environment. In cleaning up 
> one of the nodes I get the exception below. Googling around seems to reveal 
> people having trouble with it caused by too-small heap sizes but that doesn't 
> look to be what's going on here. Am I missing something obvious?
>
> $ time ./cassandra-0.7/bin/nodetool -h cassa7test01 cleanup
> Error occured while cleaning up keyspace keyspace
> java.util.concurrent.ExecutionException: java.io.IOException: Keys must be 
> written in ascending order.
>        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
>        at java.util.concurrent.FutureTask.get(FutureTask.java:111)
>        at 
> org.apache.cassandra.db.CompactionManager.performCleanup(CompactionManager.java:180)
>        at 
> org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilyStore.java:909)
>        at 
> org.apache.cassandra.service.StorageService.forceTableCleanup(StorageService.java:1127)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>        at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        at java.lang.reflect.Method.invoke(Method.java:616)
>        at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
>        at 
> com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
>        at 
> com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226)
>        at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
>        at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:251)
>        at 
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:857)
>        at 
> com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:795)
>        at 
> javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1449)
>        at 
> javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:90)
>        at 
> javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1284)
>        at 
> javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1382)
>        at 
> javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:807)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>        at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>        at java.lang.reflect.Method.invoke(Method.java:616)
>        at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
>        at sun.rmi.transport.Transport$1.run(Transport.java:177)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
>        at 
> sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
>        at 
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
>        at 
> sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
>        at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>        at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>        at java.lang.Thread.run(Thread.java:636)
> Caused by: java.io.IOException: Keys must be written in ascending order.
>        at 
> org.apache.cassandra.io.sstable.SSTableWriter.beforeAppend(SSTableWriter.java:107)
>        at 
> org.apache.cassandra.io.sstable.SSTableWriter.append(SSTableWriter.java:124)
>        at 
> org.apache.cassandra.db.CompactionManager.doCleanupCompaction(CompactionManager.java:411)
>        at 
> org.apache.cassandra.db.CompactionManager.access$400(CompactionManager.java:54)
>        at 
> org.apache.cassandra.db.CompactionManager$2.call(CompactionManager.java:171)
>        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>        at java.util.concurrent.FutureTask.run(FutureTask.java:166)
>        ... 3 more
>
> real    14m27.895s
> user    0m0.670s
> sys     0m0.200s
>
>
>



-- 
Jonathan Ellis
Project Chair, Apache Cassandra
co-founder of Riptano, the source for professional Cassandra support
http://riptano.com

Reply via email to