I am running a move on one node in a 5 node cluster. There are no writes to the cluster during the move.
I am seeing an exception on one of the nodes (not the node which I am doing the move on). The exception stack is ERROR [CompactionExecutor:1] 2011-02-04 08:10:46,855 PrecompactedRow.java (line 82) Skipping row DecoratedKey(656517988577125179070965247963445, 555345524e414d452e6a6f746173696c766573747265) in /var/lib/cassandra/data/Wenzani/UUID_UUID_SUPER-e-408-Data.db java.io.EOFException at java.io.RandomAccessFile.readFully(RandomAccessFile.java:416) at org.apache.cassandra.utils.FBUtilities.readByteArray(FBUtilities.java:280) at org.apache.cassandra.db.ColumnSerializer.deserialize(ColumnSerializer.java:94) at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:364) at org.apache.cassandra.db.SuperColumnSerializer.deserialize(SuperColumn.java:313) at org.apache.cassandra.db.ColumnFamilySerializer.deserializeColumns(ColumnFamilySerializer.java:129) at org.apache.cassandra.io.sstable.SSTableIdentityIterator.getColumnFamilyWithColumns(SSTableIdentityIterator.java:137) at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:78) at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:138) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:107) at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:42) at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73) at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136) at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131) at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183) at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94) at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:323) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:122) at org.apache.cassandra.db.CompactionManager$1.call(CompactionManager.java:92) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) at java.util.concurrent.FutureTask.run(FutureTask.java:166) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:636) Output from nodetool ring. Address Status State Load Owns Token 105916716988735575505223832861775432335 1.1.1.2 Up Normal 34.29 GB 45.36% 12956529933298582072612274413196299151 1.1.1.3 Up Normal 34.46 GB 11.41% 32366675628954067180152712803029297247 1.1.1.4 Up Normal 48.96 GB 11.40% 51756081624280481651195537730585467204 1.1.1.5 Up Normal 22 GB 22.78% 90515859237527157456212262236145255573 1.1.1.6 Up Leaving 13.34 GB 9.05% 105916716988735575505223832861775432335 1.1.1.6 is the node which I executed the move on. It seems to be locked in the Leaving state. Is this normal until the move completes? There is almost no activity in the logs and very little cpu usage across the cluster. Is this expected for a move? Cheers Stu
