Lol, yup, I just found out that is the issue!!!! Thanks, Dean On 5/24/13 8:55 AM, "Yuki Morishita" <mor.y...@gmail.com> wrote:
>hmm, I only can say it may caused by corrupt SSTable... >Stream hang on unexpected error was fixed in 1.2.5 >(https://issues.apache.org/jira/browse/CASSANDRA-5229). > >On Fri, May 24, 2013 at 6:56 AM, Hiller, Dean <dean.hil...@nrel.gov> >wrote: >> The exception on that node was just this >> >> ERROR [Thread-6056] 2013-05-22 14:47:59,416 CassandraDaemon.java (line >> 132) Exception in thread Thread[Thread-6056,5,main] >> java.lang.IndexOutOfBoundsException >> at >>sun.nio.ch.ChannelInputStream.read(ChannelInputStream.java:75) >> at >> >>org.apache.cassandra.streaming.compress.CompressedInputStream$Reader.runM >>ay >> Throw(CompressedInputStream.java:151) >> at >> org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) >> at java.lang.Thread.run(Thread.java:662) >> >> >> >> >> On 5/23/13 9:51 AM, "Yuki Morishita" <mor.y...@gmail.com> wrote: >> >>>What kind of error does the other end of streaming(/10.10.42.36) say? >>> >>>On Wed, May 22, 2013 at 5:19 PM, Hiller, Dean <dean.hil...@nrel.gov> >>>wrote: >>>> We had 3 nodes roll on good and the next 2, we see a remote node with >>>>this exception every time we start over and bootstrap the node >>>> >>>> ERROR [Streaming to /10.10.42.36:2] 2013-05-22 14:47:59,404 >>>>CassandraDaemon.java (line 132) Exception in thread Thread[Streaming to >>>>/10.10.42.36:2,5,main] >>>> java.lang.RuntimeException: java.io.IOException: Input/output error >>>> at >>>>com.google.common.base.Throwables.propagate(Throwables.java:160) >>>> at >>>>org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:32) >>>> at >>>>java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecut >>>>or >>>>.java:895) >>>> at >>>>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.j >>>>av >>>>a:918) >>>> at java.lang.Thread.run(Thread.java:662) >>>> Caused by: java.io.IOException: Input/output error >>>> at sun.nio.ch.FileChannelImpl.transferTo0(Native Method) >>>> at >>>>sun.nio.ch.FileChannelImpl.transferToDirectly(FileChannelImpl.java:405) >>>> at >>>>sun.nio.ch.FileChannelImpl.transferTo(FileChannelImpl.java:506) >>>> at >>>>org.apache.cassandra.streaming.compress.CompressedFileStreamTask.stream >>>>(C >>>>ompressedFileStreamTask.java:90) >>>> at >>>>org.apache.cassandra.streaming.FileStreamTask.runMayThrow(FileStreamTas >>>>k. >>>>java:91) >>>> at >>>>org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) >>>> ... 3 more >>>> >>>> Are there any ideas what this is? Google doesn't real show any useful >>>>advice on this and our node has not joined the ring yet so I don't >>>>think >>>>we can run a repair just yet to avoid it and try synching via another >>>>means. It seems on a streaming failure, it never recovers from this. >>>>Any ideas? >>>> >>>> We are on cassandra 1.2.2 >>>> >>>> Thanks, >>>> Dean >>>> >>> >>> >>> >>>-- >>>Yuki Morishita >>> t:yukim (http://twitter.com/yukim) >> > > > >-- >Yuki Morishita > t:yukim (http://twitter.com/yukim)
