I would assume that's 85 GB (i.e. gigabytes) then. Which is
approximately 79 GiB (i.e. gibibytes). This still sounds awfully slow -
less than 1MB/s over a full day (24 hours).
You said CPU and network aren't the bottleneck. Have you checked the
disk IO? Also, be mindful with CPU usage. It can still be a bottleneck
if one thread uses 100% of a CPU core while all other cores are idle.
On 08/07/2022 07:09, Marc Hoppins wrote:
Thank you for pointing that out.
85 gigabytes/gibibytes/GIGABYTES/GIBIBYTES/whatever name you care to give it
CPU and bandwidth are not the problem.
Version 4.0.3 but, as I stated, all nodes use the same version so the version
is not important either.
Existing nodes have 350-400+(choose whatever you want to call a gigabyte)
The problem appears to be that adding new nodes is a serial process, which is
fine when there is no data and each node is added within 2minutes. It is
hardly practical in production.
-----Original Message-----
From: Bowen Song via user <user@cassandra.apache.org>
Sent: Thursday, July 7, 2022 8:43 PM
To: user@cassandra.apache.org
Subject: Re: Adding nodes
EXTERNAL
86Gb (that's gigabits, which is 10.75GB, gigabytes) took an entire day seems
obviously too long. I would check the network bandwidth, disk IO and CPU usage
and find out what is the bottleneck.
On 07/07/2022 15:48, Marc Hoppins wrote:
Hi all,
Cluster of 2 DC and 24 nodes
DC1 (RF3) = 12 nodes, 16 tokens each
DC2 (RF3) = 12 nodes, 16 tokens each
Adding 12 more nodes to DC1: I installed Cassandra (version is the same across
all nodes) but, after the first node added, I couldn't seem to add any further
nodes.
I check nodetool status and the newly added node is UJ. It remains this way all
day and only 86Gb of data is added to the node over the entire day (probably
not yet complete). This seems a little slow and, more than a little
inconvenient to only be able to add one node at a time - or at least one node
every 2 minutes. When the cluster was created, I timed each node from service
start to status UJ (having a UUID) and it was around 120 seconds. Of course
there was no data.
Is it possible I have some setting not correctly tuned?
Thanks
Marc
