> I was able to scrub the node the repair that failed was running on. Are you > saying the error could be displayed on that node but the bad data coming from > another node ? Yes. The error occurred the node was receiving a data stream from another, you will need to clean the source of the data. You can either crawl through the logs or scrub the entire cluster.
Cheers ----------------- Aaron Morton Freelance Developer @aaronmorton http://www.thelastpickle.com On 4/01/2012, at 9:15 AM, Philippe wrote: > I was able to scrub the node the repair that failed was running on. Are you > saying the error could be displayed on that node but the bad data coming from > another node ? > > Log inspection also showed many of these, they seem to happen around when a > stream transfer finishes. > ERROR [Thread-550876] 2012-01-03 16:35:31,922 AbstractCassandraDaemon.java > (line 139) Fatal exception in thread Thread[Thread-550876,5,main] > java.lang.IllegalArgumentException > at sun.nio.ch.FileChannelImpl.transferFrom(FileChannelImpl.java:586) > at > org.apache.cassandra.streaming.IncomingStreamReader.readnwrite(IncomingStreamReader.java:110) > at > org.apache.cassandra.streaming.IncomingStreamReader.readFile(IncomingStreamReader.java:85) > at > org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:61) > at > org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189) > at > org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117) > > Thanks > > 2012/1/2 aaron morton <aa...@thelastpickle.com> > I would try to nodetool scrub the data on the node that that sent the bad > data in the stream. You may be able to work which node from the logs, or it > may be easier to just scrub them all. > > Hope that helps. > > ----------------- > Aaron Morton > Freelance Developer > @aaronmorton > http://www.thelastpickle.com > > On 31/12/2011, at 12:20 AM, Philippe wrote: > >> Hello, >> Running a combination of 0.8.6 and 0.8.8 with RF=3, I am getting the >> following while repairing one node (all other nodes completed successfully). >> Can I just stop the instance, erase the SSTable and restart cleanup ? >> Thanks >> >> ERROR [Thread-402484] 2011-12-29 14:51:03,687 AbstractCassandraDaemon.java >> (line 139) Fatal exception in thread Thread[Thread-402484,5,main] >> java.lang.RuntimeException: java.util.concurrent.ExecutionException: >> java.io.IOError: java.io.IOException: Corrupt (negative) value length >> encountered >> at >> org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:154) >> at >> org.apache.cassandra.streaming.IncomingStreamReader.read(IncomingStreamReader.java:63) >> at >> org.apache.cassandra.net.IncomingTcpConnection.stream(IncomingTcpConnection.java:189) >> at >> org.apache.cassandra.net.IncomingTcpConnection.run(IncomingTcpConnection.java:117) >> Caused by: java.util.concurrent.ExecutionException: java.io.IOError: >> java.io.IOException: Corrupt (negative) value length encountered >> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222) >> at java.util.concurrent.FutureTask.get(FutureTask.java:83) >> at >> org.apache.cassandra.streaming.StreamInSession.closeIfFinished(StreamInSession.java:138) >> ... 3 more >> > >
