This is the log of the destination/rebuilding node, you need to check what is the error message on the stream source node (192.168.1.140).
2016-05-25 15:22 GMT-03:00 George Sigletos <sigle...@textkernel.nl>: > Hello, > > Here is additional stack trace from system.log: > > ERROR [STREAM-IN-/192.168.1.140] 2016-05-24 22:44:57,704 > StreamSession.java:620 - [Stream #2c290460-20d4-11e6-930f-1b05ac77baf9] > Remote peer 192.168.1.140 failed stream session. > ERROR [STREAM-OUT-/192.168.1.140] 2016-05-24 22:44:57,705 > StreamSession.java:505 - [Stream #2c290460-20d4-11e6-930f-1b05ac77baf9] > Streaming error occurred > java.io.IOException: Connection timed out > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) > ~[na:1.7.0_79] > at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_79] > at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) > ~[na:1.7.0_79] > at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_79] > at sun.nio.ch.SocketChannelImpl.write(Unknown Source) > ~[na:1.7.0_79] > at > org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:351) > [apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:323) > [apache-cassandra-2.1.13.jar:2.1.13] > at java.lang.Thread.run(Unknown Source) [na:1.7.0_79] > INFO [STREAM-IN-/192.168.1.140] 2016-05-24 22:44:58,625 > StreamResultFuture.java:180 - [Stream > #2c290460-20d4-11e6-930f-1b05ac77baf9] Session with /192.168.1.140 is > complete > WARN [STREAM-IN-/192.168.1.140] 2016-05-24 22:44:58,627 > StreamResultFuture.java:207 - [Stream > #2c290460-20d4-11e6-930f-1b05ac77baf9] Stream failed > ERROR [RMI TCP Connection(24)-127.0.0.1] 2016-05-24 22:44:58,628 > StorageService.java:1075 - Error while rebuilding node > org.apache.cassandra.streaming.StreamException: Stream failed > at > org.apache.cassandra.streaming.management.StreamEventJMXNotifier.onFailure(StreamEventJMXNotifier.java:85) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > com.google.common.util.concurrent.Futures$4.run(Futures.java:1172) > ~[guava-16.0.jar:na] > at > com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297) > ~[guava-16.0.jar:na] > at > com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156) > ~[guava-16.0.jar:na] > at > com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145) > ~[guava-16.0.jar:na] > at > com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:202) > ~[guava-16.0.jar:na] > at > org.apache.cassandra.streaming.StreamResultFuture.maybeComplete(StreamResultFuture.java:208) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.StreamResultFuture.handleSessionComplete(StreamResultFuture.java:184) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.StreamSession.closeSession(StreamSession.java:415) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.StreamSession.sessionFailed(StreamSession.java:621) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.StreamSession.messageReceived(StreamSession.java:475) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.ConnectionHandler$IncomingMessageHandler.run(ConnectionHandler.java:256) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at java.lang.Thread.run(Unknown Source) ~[na:1.7.0_79] > ERROR [STREAM-OUT-/192.168.1.140] 2016-05-24 22:44:58,629 > StreamSession.java:505 - [Stream #2c290460-20d4-11e6-930f-1b05ac77baf9] > Streaming error occurred > java.io.IOException: Broken pipe > at sun.nio.ch.FileDispatcherImpl.write0(Native Method) > ~[na:1.7.0_79] > at sun.nio.ch.SocketDispatcher.write(Unknown Source) ~[na:1.7.0_79] > at sun.nio.ch.IOUtil.writeFromNativeBuffer(Unknown Source) > ~[na:1.7.0_79] > at sun.nio.ch.IOUtil.write(Unknown Source) ~[na:1.7.0_79] > at sun.nio.ch.SocketChannelImpl.write(Unknown Source) > ~[na:1.7.0_79] > at > org.apache.cassandra.io.util.DataOutputStreamAndChannel.write(DataOutputStreamAndChannel.java:48) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.messages.StreamMessage.serialize(StreamMessage.java:44) > ~[apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.sendMessage(ConnectionHandler.java:351) > [apache-cassandra-2.1.13.jar:2.1.13] > at > org.apache.cassandra.streaming.ConnectionHandler$OutgoingMessageHandler.run(ConnectionHandler.java:331) > [apache-cassandra-2.1.13.jar:2.1.13] > at java.lang.Thread.run(Unknown Source) [na:1.7.0_79] > > > On Wed, May 25, 2016 at 5:23 PM, Paulo Motta <pauloricard...@gmail.com> > wrote: > >> The stack trace from the rebuild command not show the root cause of the >> rebuild stream error. Can you check the system.log for ERROR logs during >> streaming and paste here? >> > >
