Spot on, once I upgraded the client to 15.2.8 I was able to enable rbd-mirror snapshots and create them while the VM was running.
However, I have noticed that I am also able to break replication when the rbd is being used by a KVM VM. While writing data in the VM, I took a rbd snapshot, then kicked off a mirror snapshot. I took the rbd snapshot and mirror snapshot from one the osd nodes. Primary vm-101-disk-0: global_id: 68fe424e-3683-48af-ba39-2b0b4726d918 state: up+stopped description: local image is primary service: admin on Ccscephmon1 last_update: 2021-01-29 10:44:40 peer_sites: name: bunker state: up+error description: failed to copy snapshots from remote to local image last_update: 2021-01-29 10:44:50 root@Ccscephtest1:~# rbd snap ls --all CephTestPool1/vm-101-disk-0 SNAPID NAME SIZE PROTECTED TIMESTAMP NAMESPACE 1581 .mirror.primary.68fe424e-3683-48af-ba39-2b0b4726d918.3f9da1d2-7f23-4a75-865e-3af062be93be 2 TiB Fri Jan 29 10:28:53 2021 mirror (primary peer_uuids:[35c97c1f-a897-4635-abf6-dbd4085c89bd]) 1584 Test 2 TiB Fri Jan 29 10:37:37 2021 user 1585 .mirror.primary.68fe424e-3683-48af-ba39-2b0b4726d918.24744afa-2d9f-4958-8775-c23b0b7f383c 2 TiB Fri Jan 29 10:37:47 2021 mirror (primary peer_uuids:[35c97c1f-a897-4635-abf6-dbd4085c89bd]) Secondary vm-101-disk-0: global_id: 68fe424e-3683-48af-ba39-2b0b4726d918 state: up+error description: failed to copy snapshots from remote to local image service: admin on Bunkcephtest1 last_update: 2021-01-29 10:47:12 peer_sites: name: ccs state: up+stopped description: local image is primary last_update: 2021-01-29 10:47:10 root@Bunkcephtest1:~# rbd snap ls --all CephTestPool1/vm-101-disk-0 SNAPID NAME SIZE PROTECTED TIMESTAMP NAMESPACE 1552 Test 2 TiB Fri Jan 29 10:28:53 2021 user 1553 .mirror.non_primary.68fe424e-3683-48af-ba39-2b0b4726d918.0cd471db-7ed7-4700-8da0-8a82bc2d23a5 2 TiB Fri Jan 29 10:28:54 2021 mirror (non-primary peer_uuids:[] adcedc94-b1a1-4508-8402-db4ec0dc3085:1581 copied) I tried restarting the rbd-mirror services but no luck. I ended up issuing a resync and that got things moving until I repeated the steps again. I have the option of using KRBD but not sure if that will help in this situation. From: "Jason Dillaman" <jdill...@redhat.com> To: "adamb" <ad...@medent.com> Cc: "ceph-users" <ceph-users@ceph.io> Sent: Friday, January 29, 2021 9:39:28 AM Subject: Re: [ceph-users] Unable to enable RBD-Mirror Snapshot on image when VM is using RBD On Fri, Jan 29, 2021 at 9:34 AM Adam Boyhan <ad...@medent.com> wrote: > > This is a odd one. I don't hit it all the time so I don't think its expected > behavior. > > Sometimes I have no issues enabling rbd-mirror snapshot mode on a rbd when > its in use by a KVM VM. Other times I hit the following error, the only way I > can get around it is to power down the KVM VM. > > root@Ccscephtest1:~# rbd mirror image enable CephTestPool1/vm-101-disk-0 > snapshot > 2021-01-29T09:29:07.875-0500 7f1e99ffb700 -1 > librbd::mirror::snapshot::CreatePrimaryRequest: 0x7f1e7c012440 > handle_create_snapshot: failed to create mirror snapshot: (22) Invalid > argument > 2021-01-29T09:29:07.875-0500 7f1e99ffb700 -1 librbd::mirror::EnableRequest: > 0x5597667fd200 handle_create_primary_snapshot: failed to create initial > primary snapshot: (22) Invalid argument > 2021-01-29T09:29:07.875-0500 7f1ea559f3c0 -1 librbd::api::Mirror: > image_enable: cannot enable mirroring: (22) Invalid argument I suspect that you have the exclusive-lock feature enabled and the QEMU hypervisor host has a pre-octopus version of librbd. > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > -- Jason _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
