[ceph-users] rbd-mirror keeps crashing

2024-08-01 Thread armsby
Hi everyone, I've been running rbd-mirror between my old Ceph system (16.2.10) and my new system (18.2.2). I'm using journaling mode on a pool that contains 7,500 images. Everything was running perfectly until it processed about 5,608 images. Now, it keeps crashing with the following message:

[ceph-users] RBD Mirror - Failed to unlink peer

2024-05-30 Thread Scott Cairns
Hi, Following the introduction of an additional node to our Ceph cluster, we've started to see unlink errors when taking a rbd mirror snapshot. We've had RBD mirroring configured for over a year now and it's been working flawlessly, however after we created OSD's on a new node we've receiving t

[ceph-users] RBD Mirror - Failed to unlink peer

2024-05-29 Thread scott . cairns
I've had rbd mirror deployed for over a year copying images from Site A to Site B for DR. After introducing an additional node into the cluster a few weeks back and creating OSD's on this node (no mon/mgr) we've started to see snapshots throwing an error 'librbd::mirror::snapshot::CreatePrimary

[ceph-users] RBD Mirror - implicit snapshot cleanup

2024-05-29 Thread scott . cairns
With RBD Mirroring configured from Site A to Site B, we typically see 2 snapshots on the source side (current and 1 previous) and 1 snapshot on the destination side (most recent snapshot from source) - this has been the case for over a year now. When testing a DR scenario (i.e. demoting the pri

[ceph-users] [rbd mirror] integrity of journal-based image mirror

2024-05-27 Thread Tony Liu
Hi, Say, the source image is being updated and data is mirrored to destination continuously. Suddenly, networking of source is down and destination will be promoted and used to restore the VM. Is that going to cause any FS issue and, for example, fsck needs to be invoked to check and repair FS?

[ceph-users] rbd-mirror get status updates quicker

2024-04-25 Thread Stefan Kooman
Hi, We're testing with rbd-mirror (mode snapshot) and try to get status updates about snapshots as fast a possible. We want to use rbd-mirror as a migration tool between two clusters and keep downtime during migration as short as possible. Therefore we have tuned the following parameters and

[ceph-users] rbd-mirror failed to query services: (13) Permission denied

2024-04-22 Thread Stefan Kooman
Hi, We are testing rbd-mirroring. There seems to be a permission error with the rbd-mirror user. Using this user to query the mirror pool status gives: failed to query services: (13) Permission denied And results in the following output: health: UNKNOWN daemon health: UNKNOWN image health: O

[ceph-users] rbd-mirror and DR test

2023-09-18 Thread Kamil Madac
One of our customers is currently facing a challenge in testing our disaster recovery (DR) procedures on a pair of Ceph clusters (Quincy version 17.2.5). Our issue revolves around the need to resynchronize data after conducting a DR procedure test. In small-scale scenarios, this may not be a signi

[ceph-users] [rbd-mirror] can't enable journal-based image mirroring

2023-07-31 Thread Tony Liu
Hi, The Ceph cluster is with Pacific v16.2.10. "rbd mirror image enable journal" seems not working. Any clues what I'm missing? There is no error messages from the CLI. Any way to troubleshooting? ``` # rbd mirror pool info volume-ssd Mode: image Site Name: 35d050c0-77c0-11eb-9242-2cea7ff9d07c

[ceph-users] rbd mirror snapshot trash

2023-05-16 Thread Eugen Block
Good morning, I would be grateful if anybody could shed some light on this, I can't reproduce it in my lab clusters so I was hoping for the community. A customer has 2 clusters with rbd mirroring (snapshots) enabled, it seems to work fine, they have regular checks and the images on the remo

[ceph-users] rbd-mirror replication speed is very slow - but initial replication is fast

2023-01-29 Thread ankit raikwar
Hello Team, Please help me i deploy two ceph cluster with 6 node configuration almost 800tb of capacity. and configurae in the DC-DR configuration for the data high availability. i eanbel the rwg and rbd block device mirroring for the replocatio of the data. we have the 10 GBPS fiber replication

[ceph-users] rbd-mirror | ceph quincy Not able to find rbd_mirror_journal_max_fetch_bytes config in rbd mirror

2023-01-16 Thread ankit raikwar
Hello All, In the ceph quincy Not able to find rbd_mirror_journal_max_fetch_bytes config in rbd mirror i configured the ceph cluster almost 400 tb and enable the rbd-mirror in the starting stage i'm able to achive the almost 9 GB speed , but after the rebalane completed

[ceph-users] rbd-mirror ceph quincy Not able to find rbd_mirror_journal_max_fetch_bytes config in rbd mirror

2023-01-12 Thread ankit raikwar
Hello All, In the ceph quincy Not able to find rbd_mirror_journal_max_fetch_bytes config in rbd mirror i configured the ceph cluster almost 400 tb and enable the rbd-mirror in the starting stage i'm able to achive the almost 9 GB speed , but after the rebalane completed

[ceph-users] rbd-mirror stops replaying journal on primary cluster

2022-08-01 Thread Josef Johansson
On Mon, Aug 1, 2022 at 2:33 PM Josef Johansson wrote: > > > > > Forwarded Message > Subject:rbd-mirror stops replaying journal on primary cluster > Date: Thu, 2 Jun 2022 16:29:33 +0200 (CEST) > From: Josef Johansson > To: ceph-users@ceph.io > > > > Hi, > I'm run

[ceph-users] RBD mirror direction settings issue

2022-04-30 Thread Denis Polom
Hi, I'm setting up RBD mirror between two Ceph clusters and have an issue to set up rx-tx direction on primary site. Issuing the command rbd mirror pool peer bootstrap import --direction rx-tx --site-name primary rbd token I'm expecting the bi-directional mirror for the pool rbd. But I'm

[ceph-users] rbd mirror between clusters with private "public" network

2022-04-25 Thread Tony Liu
Hi, I understand that, for rbd mirror to work, the rbd mirror service requires the connectivity to all nodes from both cluster. In my case, for security purpose, the "public" network is actually a private network, which is not routable to external. All internal RBD clients are on that private n

[ceph-users] Rbd mirror

2021-09-30 Thread Ignazio Cassano
Hello All, Please I would like to know if it is possibile two clusters can mirror rbd to a third cluster. In other words I have 3 separated ceph cluster : A B C. I would like cluster A and cluster B can mirror some pools on cluster C. Is it possible ? Thanks Ignazio ___

[ceph-users] RBD-Mirror Snapshot Scalability

2021-01-21 Thread Adam Boyhan
I have noticed that RBD-Mirror snapshot mode can only manage to take 1 snapshot per second. For example I have 21 images in a single pool. When the schedule is triggered it takes the mirror snapshot of each image 1 at a time. It doesn't feel or look like a performance issue as the OSD's are Micr

[ceph-users] RBD-Mirror Mirror Snapshot stuck

2021-01-21 Thread Adam Boyhan
I have a rbd-mirror snapshot on 1 image that failed to replicate and now its not getting cleaned up. The cause of this was my fault based on my steps. Just trying to understand how to clean up/handle the situation. Here is how I got into this situation. - Created manual rbd snapshot on the

[ceph-users] RBD-Mirror Snapshot Backup Image Uses

2021-01-20 Thread Adam Boyhan
I have been doing some testing with RBD-Mirror Snapshots to a remote Ceph cluster. Does anyone know if the images on the remote cluster can be utilized in anyway? Would love the ability to clone them, or even readonly would be nice. ___ ceph-users ma

[ceph-users] RBD-Mirror: snapshots automatically created?

2020-09-22 Thread Eugen Block
Hi *, I encountered a rather strange (or at least unexpected) behaviour of the rbd-mirror. Maybe I don't fully understand the feature so please correct me if my assumptions are wrong. My two (virtual one-node) clusters are still on ceph version 15.2.4-864-g0f510cb110 and the following hap

[ceph-users] rbd-mirror with snapshot, not doing any actaul data sync

2020-06-07 Thread Hans van den Bogert
Hi list, I've awaited octopus for a along time to be able to use mirror with snapshotting, since my setup does not allow for journal based mirroring. (K8s/Rook 1.3.x with ceph 15.2.2) However, I seem to be stuck, i've come to the point where on the cluster on which the (non-active) replicas shoul

[ceph-users] rbd-mirror sync image continuously or only sync once

2020-06-03 Thread Zhenshi Zhou
Hi all, I'm gonna deploy a rbd-mirror in order to sync image from clusterA to clusterB. The image will be used while syncing. I'm not sure if the rbd-mirror will sync image continuously or not. If not, I will inform clients not to write data in it. Thanks. Regards

[ceph-users] rbd-mirror -> how far behind_master am i time wise?

2020-03-24 Thread Ml Ml
Hello List, i use rbd-mirror and i asynchronously mirror to my backup cluster. My backup cluster only has "spinnung rust" and wont be able to always perform like the live cluster. Thats is fine for me, as far as it´s not further behind than 12h. vm-194-disk-1: global_id: 7a95730f-451c-4973-8

[ceph-users] rbd-mirror replay is very slow - but initial bootstrap is fast

2020-03-10 Thread Ml Ml
Hello List, when i initially enable journal/mirror on an image it gets bootstrapped to my site-b pretty quickly with 250MB/sec which is about the IO Write limit. Once its up2date, the replay is very slow. About 15KB/sec and the entries_behind_maste is just running away: root@ceph01:~# rbd --clus

[ceph-users] rbd-mirror - which direction?

2020-03-05 Thread Ml Ml
Hello, i am running luminous and i would like to back up my cluster from Site-A to Site-B (one way) So i decided to mirror it to an off-site ceph cluster. I read: https://docs.ceph.com/docs/luminous/rbd/rbd-mirroring/ But i liked I https://github.com/MiracleMa/Blog/issues/2 a little better. Bu

[ceph-users] RBD-mirror instabilities

2019-12-23 Thread Oliver Freyermuth
Dear Cephalopodians, running 13.2.6 on the source cluster and 14.2.5 on the rbd mirror nodes and the target cluster, I observe regular failures of rbd-mirror processes. With failures, I mean that traffic stops, but the daemons are still listed as active rbd-mirror daemons in "ceph -s", and the

[ceph-users] RBD Mirror, Clone non primary Image

2019-10-21 Thread yveskretzschmar
Hi, I have a working RBD Mirror Setup using ceph version 14.2.4 on both sides. I want to have a clone of a non primay image. I do it this way: 1. create snapshot of primary image 2. wait for the snapshot to appear on the backup cluster 3. create a clone in backup cluster (using Simplified RBD Im