Hinted hand off is a best effort approach, and relying on it alone is a
bad idea. Hints can get lost due to a number of reasons, such as getting
too old or too big, or the node storing the hints dies. You should rely
on regular repair to guarantee the correctness of the data. You may use
hinted hand off as a supplement, but not a replacement.
On 05/11/2024 15:53, edi mari wrote:
Thank you for your reply, Bowen.
Correct, the questions were about migrating the server hardware to a
new location, not the Cassandra DC.
Wouldn’t it be a good idea to use the hints to complete the data to DC3?
I'll extend the hint window (e.g., to one week) and allow the other
data centers (DC1 and DC2) to save hints for DC3.
Then, when DC3 returns online, it can receive and process the hints.
Edi
On Tue, Nov 5, 2024 at 2:34 PM Bowen Song via user
<user@cassandra.apache.org> wrote:
You just confirmed my suspicion. You are indeed referring to both
physical location of servers and the logical Cassandra DC with the
same term here.
The questions are related to the procedure of migrating the server
hardware to a new location, not the Cassandra DC.
Assuming that the IP addresses of servers don't change, and all
data on all servers can be preserved, this process should be
fairly simple and straight forward.
I believe the best approach here is:
1. ensure the application can handle the database servers go
offline for extended period of time
(e.g. they talk to the not migrating nodes, and the query CLs
don't require nodes in the migrating DC to be available)
2. ensure the gc_grace_seconds of all tables is long enough to
complete the migration and run repair twice
3. run and finish a full repair right before the migration begins
4. migrate the server hardware to the new location
5. run a full repair again to bring the data back in sync
6. restore the gc_grace_seconds back to appropriate value
As always, please ensure you have data backups, and make a
contingency plan for potential issues in advance is also highly
recommended.
On 05/11/2024 12:00, edi mari wrote:
Each physical data center corresponds to a "logical" Cassandra DC
(a group of nodes).
In our situation, we need to move one of our physical data
centers (i.e., the server rooms) to a new location, which will
involve an extended period of downtime.
Thanks
Edi
On Tue, Nov 5, 2024 at 1:27 PM Bowen Song via user
<user@cassandra.apache.org> wrote:
From the way you wrote this, I suspect the name DC may have
different
meaning here. Are you talking about the physical location
(i.e server
rooms), or the Cassandra DC (i.e. group of nodes for
replication purposes)?
On 05/11/2024 11:01, edi mari wrote:
> Hello,
> We have a Cassandra cluster deployed across three different
data
> centers, with each data center (DC1, DC2, and DC3) hosting 50
> Cassandra nodes.
>
> We are currently saving one replica in each data center.
> We plan to migrate DC3, including storage and servers, to a
new data
> center.
>
> 1. What would be the best method to perform this migration,
which
> could take several days (2–5 days)?
> 2. Would relying on hints in the other two data centers be
a good
> approach?
>
> Thanks
> Edi
