Is there any chance you've set streaming_socket_timeout_in_ms parameter set too low on failing node?
-----Original Message----- From: Micha [mailto:mich...@fantasymail.de] Sent: Tuesday, August 08, 2017 3:01 AM To: user@cassandra.apache.org; d...@cassandra.apache.org Subject: rebuild constantly fails, 3.11 Hi, it seems I'm not able to add add 3 node dc to a 3 node dc. After starting the rebuild on a new node, nodetool netstats show it will receive 1200 files from node-1 and 5000 from node-2. The stream from node-1 completes but the stream from node-2 allways fails, after sending ca 4000 files. After restarting the rebuild it again starts to send the 5000 files. The whole cluster is connected via one switch only , no firewall between, the networks shows no errors. The machines have 8 cores, 32GB RAM and two 1TB discs as raid0. the logs show no errors. The size of the data is ca 1TB. Any help is really welcome, cheers Michael The error is: Cassandra has shutdown. error: null -- StackTrace -- java.io.EOFException at java.io.DataInputStream.readByte(DataInputStream.java:267) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:222) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl_Stub.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.invoke(RMIConnector.java:1020) at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:298) at com.sun.proxy.$Proxy7.rebuild(Unknown Source) at org.apache.cassandra.tools.NodeProbe.rebuild(NodeProbe.java:1190) at org.apache.cassandra.tools.nodetool.Rebuild.execute(Rebuild.java:58) at org.apache.cassandra.tools.NodeTool$NodeToolCmd.run(NodeTool.java:254) at org.apache.cassandra.tools.NodeTool.main(NodeTool.java:168) --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@cassandra.apache.org For additional commands, e-mail: user-h...@cassandra.apache.org
