Hi Vsevolod, > Are there any workarounds to speed up the process? (e.g. doing cleanup only > after all 4 new nodes joined cluster), or inserting multiple nodes > simultaneously with specific settings? e.g. doing cleanup only after all 4 new nodes joined cluster === allowed inserting multiple nodes simultaneously with specific settings ==== generally, possible, but not recommended. Add/remove nodes one by one if needed. See http://thelastpickle.com/blog/2017/05/23/auto-bootstrapping-part1.html for details also you may play with stream_throughput_outbound_megabits_per_sec parameter in cassandra.yaml
> How do people with tens of Cassandra nodes perform insertion/extraction of > new/failed nodes? What's the usual routine in case you have 20+ nodes, and > need to decommission 4 nodes and insert 4 new ones instead? It’s better to replace existing node w/o streaming, just copying data by rsync / attaching EBS volume to new server etc http://thelastpickle.com/blog/2018/02/21/replace-node-without-bootstrapping.html Regards, Kyrill From: Vsevolod Filaretov <vsfilare...@gmail.com> Reply-To: "user@cassandra.apache.org" <user@cassandra.apache.org> Date: Tuesday, March 12, 2019 at 11:01 To: "user@cassandra.apache.org" <user@cassandra.apache.org> Subject: New node insertion methods Hello everyone! We have a cluster of 4 nodes, 4.5 tb/data per node, and are in the middle of adding 4 more nodes to the cluster. Joining a new node based on official guidelines/helps (setup cassandra on a new node, start cassandra instance, wait until node goes from JOINING state to NORMAL, perform nodetool cleanup) takes approximately 7 days in our case, per single node. Questions: 1) Are there any workarounds to speed up the process? (e.g. doing cleanup only after all 4 new nodes joined cluster), or inserting multiple nodes simultaneously with specific settings? 2) How do people with tens of Cassandra nodes perform insertion/extraction of new/failed nodes? What's the usual routine in case you have 20+ nodes, and need to decommission 4 nodes and insert 4 new ones instead? Links to blog posts or mail threads much appreciated! Thank you all in advance, Vsevolod Filaretov.
