Hi.
I'm running 5 cassandra nodes. Say, the addresses are 112.234.123.111 ~
112.234.123.115; the real address is different.
When I run nodetool, the one node of address 112.234.123.112 has failed to
connect. Showing error message like this.
iPark:~ hayarobi$ nodetool --host 112.234.123.112 ring
Error connection to remote JMX agent!
java.rmi.ConnectException: Connection refused to host: 122.234.123.112; nested
exception is:
The host to connect address differ! I had tried to query 112.* but, the
nodetool tried to connect 122.*. It happened just one machine. All other
machines works fine.
And I can connect to 112.234.123.112 by cassandra-cli or other tools using
other port (such as 22 of ssh, 80 of http). It has trouble only on nodetool.
Does anyone has an idea?
I'll paste the full stack trace below.
iPark:~ hayarobi$ nodetool --host 112.234.123.111 ring
Address Status State Load Owns Token
136112946768375000000000000000000000000
112.234.123.111 Up Normal 725.01 KB 20.00% 0
112.234.123.112 Up Normal 725.93 KB 20.00%
34028236692093800000000000000000000000
112.234.123.113 Up Normal 728.2 KB 20.00%
68056473384187700000000000000000000000
112.234.123.114 Up Normal 713.1 KB 20.00%
102084710076282000000000000000000000000
112.234.123.115 Up Normal 722.67 KB 20.00%
136112946768375000000000000000000000000
iPark:~ hayarobi$ nodetool --host 112.234.123.115 ring
Address Status State Load Owns Token
136112946768375000000000000000000000000
112.234.123.111 Up Normal 725.01 KB 20.00% 0
112.234.123.112 Up Normal 725.93 KB 20.00%
34028236692093800000000000000000000000
112.234.123.113 Up Normal 728.2 KB 20.00%
68056473384187700000000000000000000000
112.234.123.114 Up Normal 713.1 KB 20.00%
102084710076282000000000000000000000000
112.234.123.115 Up Normal 722.67 KB 20.00%
136112946768375000000000000000000000000
iPark:~ hayarobi$ nodetool --host 112.234.123.114 ring
Address Status State Load Owns Token
136112946768375000000000000000000000000
112.234.123.111 Up Normal 725.01 KB 20.00% 0
112.234.123.112 Up Normal 725.93 KB 20.00%
34028236692093800000000000000000000000
112.234.123.113 Up Normal 728.2 KB 20.00%
68056473384187700000000000000000000000
112.234.123.114 Up Normal 713.1 KB 20.00%
102084710076282000000000000000000000000
112.234.123.115 Up Normal 722.67 KB 20.00%
136112946768375000000000000000000000000
iPark:~ hayarobi$ nodetool --host 112.234.123.112 ring
Error connection to remote JMX agent!
java.rmi.ConnectException: Connection refused to host: 122.234.123.112; nested
exception is:
java.net.ConnectException: Connection refused
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:601)
at
sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:198)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:110)
at javax.management.remote.rmi.RMIServerImpl_Stub.newClient(Unknown
Source)
at
javax.management.remote.rmi.RMIConnector.getConnection(RMIConnector.java:2327)
at
javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:279)
at
javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:248)
at org.apache.cassandra.tools.NodeProbe.connect(NodeProbe.java:137)
at org.apache.cassandra.tools.NodeProbe.<init>(NodeProbe.java:107)
at org.apache.cassandra.tools.NodeCmd.main(NodeCmd.java:511)
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:432)
at java.net.Socket.connect(Socket.java:529)
at java.net.Socket.connect(Socket.java:478)
at java.net.Socket.<init>(Socket.java:375)
at java.net.Socket.<init>(Socket.java:189)
at
sun.rmi.transport.proxy.RMIDirectSocketFactory.createSocket(RMIDirectSocketFactory.java:22)
at
sun.rmi.transport.proxy.RMIMasterSocketFactory.createSocket(RMIMasterSocketFactory.java:128)
at sun.rmi.transport.tcp.TCPEndpoint.newSocket(TCPEndpoint.java:595)
... 10 more
