Hi
Cassandra version: 2.1.11
The existed cluster has three nodes:
[root@report-02 cassandra]# bin/nodetool status
UN 192.21.0.135 120.85 GB 512 ?
11e1e80f-9c5f-4f7c-81f2-42d3b704d8e3 RAC1
UN 192.21.0.133 129.13 GB 512 ?
3e662ccb-fa2b-427b-9ca1-c2d3468bfbc9 RAC1
UN 192.21.0.131 149.05 GB 512 ?
60f763f3-09bc-4d6f-9301-494c93857fc1 RAC1
I wanted to add two nodes and set the same configs as the cluster's nodes.
node1: 192.21.0.184
node2: 192.21.0.185
After starting the two nodes one by one, the first node 192.21.0.184 finished
the joining immediately, but the second one 192.21.0.185 took more than 24
hours to join and not finished now:
Under 192.168.0.184:
[root@report-01 cassandra]# bin/nodetool compactionstats
pending tasks: 0
Under 192.168.0.185:
[root@report-02 cassandra]# bin/nodetool compactionstats
pending tasks: 21
compaction type keyspace table completed
total unit progress
Compaction testforuser users1027 6204396079 14923537640
bytes 41.57%
Compaction user_center users 19325435997 514143044706
bytes 3.76%
Compaction user_center users 12305639479 118703090319
bytes 10.37%
Active compaction remaining time : 10h05m54s
And:
[root@report-02 cassandra]# bin/nodetool status
Datacenter: DC1
===============
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
-- Address Load Tokens Owns Host ID
Rack
UN 192.21.0.135 120.85 GB 512 ?
11e1e80f-9c5f-4f7c-81f2-42d3b704d8e3 RAC1
UN 192.21.0.133 129.13 GB 512 ?
3e662ccb-fa2b-427b-9ca1-c2d3468bfbc9 RAC1
UN 192.21.0.131 149.05 GB 512 ?
60f763f3-09bc-4d6f-9301-494c93857fc1 RAC1
UJ 192.21.0.185 299.22 GB 256 ?
84c0dd16-6491-4bfb-b288-d4e410cd8c2a RAC1
UN 192.21.0.184 670.14 MB 256 ?
4041c232-c110-4315-89a1-23ca53b851c2 RAC1
>From the above load data size, obviously, node2(192.21.0.185)'s 299.22G is
not normal.
And the node2's boostrap interrupted several times because it got a error:
INFO 00:57:42 [Stream #8eb8cbe0-c488-11e5-baf9-918c8558de90] Session
with /192.21.0.135 is complete
INFO 00:57:42 [Stream #8eb8cbe0-c488-11e5-baf9-918c8558de90] Session
with /192.21.0.131 is complete
WARN 00:57:42 [Stream #8eb8cbe0-c488-11e5-baf9-918c8558de90] Stream failed
ERROR 00:57:42 Exception encountered during startup
java.lang.RuntimeException: Error during boostrap: Stream failed
So I restarted it and the join continued!
I don't know why there is the difference between the two nodes?
I should stop it, and change something?
Thank you in advance!
Dillon
