Hi,
I would expect that you have a similar config-key entry:
ceph config-key ls |grep "peer/cephfs"
"cephfs/mirror/peer/cephfs/18c02021-8902-4e3f-bc17-eaf48331cc56",
Maybe removing that peer would already suffice?
Zitat von Jan Zeinstra <j...@delectat.nl>:
> Hi,
> This is my first post to the forum and I don't know if it's appropriate,
> but I'd like to express my gratitude to all people working hard on ceph
> because I think it's a fantastic piece of software.
>
> The problem I'm having is caused by me; we had a well working ceph fs
> mirror solution; let's call it source cluster A, and target cluster B.
> Source cluster A is a modest cluster consisting of 6 instances, 3 OSD
> instances, and 3 mon instances. The OSD instances all have 3 disks
(HDD's)
> and 3 OSD demons, totalling 9 OSD daemons and 9 HDD's. Target cluster B
is
> a single node system having 3 OSD daemons and 3 HDD's. Both clusters run
> ceph 18.2.4 reef. Both clusters use Ubuntu 22.04 as OS throughout. Both
> systems are installed using cephadm.
> I have destroyed cluster B, and have built it from the ground up (I made
a
> mistake in PG sizing in the original cluster)
> Now i find i cannot create/ reinstate the mirroring between 2 ceph fs
> filesystems, and i suspect there is a peer left behind in the filesystem
of
> the source, pointing to the now non-existent target cluster.
> When i do 'ceph fs snapshot mirror peer_list prodfs', i get:
> '{"f3ea4e15-6d77-4f28-aacb-9afbfe8cc1c5": {"client_name":
> "client.mirror_remote", "site_name": "bk-site", "fs_name": "prodfs"}}'
> When i try to delete it: 'ceph fs snapshot mirror peer_remove prodfs
> f3ea4e15-6d77-4f28-aacb-9afbfe8cc1c5', i get: 'Error EACCES: failed to
> remove peeraccess denied: does your client key have mgr caps? See
> http://docs.ceph.com/en/latest/mgr/administrator/#client-authentication
',
> but the logging of the daemon points to the more likely reason of
failure:
> ----
> Apr 08 12:54:26 s1mon systemd[1]: Started Ceph cephfs-mirror.s1mon.lvlkwp
> for d0ea284a-8a16-11ee-9232-5934f0f00ec2.
> Apr 08 12:54:26 s1mon cephfs-mirror[310088]: set uid:gid to 167:167
> (ceph:ceph)
> Apr 08 12:54:26 s1mon cephfs-mirror[310088]: ceph version 18.2.4
> (e7ad5345525c7aa95470c26863873b581076945d) reef (stable), process
> cephfs-mirror, pid 2
> Apr 08 12:54:26 s1mon cephfs-mirror[310088]: pidfile_write: ignore empty
> --pid-file
> Apr 08 12:54:26 s1mon cephfs-mirror[310088]: mgrc service_daemon_register
> cephfs-mirror.22849497 metadata
> {arch=x86_64,ceph_release=reef,ceph_version=ceph version 18.2.4
> (e7ad5345525c7a>
> Apr 08 12:54:30 s1mon cephfs-mirror[310088]:
> cephfs::mirror::PeerReplayer(f3ea4e15-6d77-4f28-aacb-9afbfe8cc1c5) init:
> remote monitor host=[v2:172.17.16.12:3300/0,v1:172.17.16.12:6789/0]
> Apr 08 12:54:30 s1mon conmon[310082]: 2025-04-08T10:54:30.365+0000
> 7f57c51ba640 -1 monclient(hunting): handle_auth_bad_method server
> allowed_methods [2] but i only support [2,1]
> Apr 08 12:54:30 s1mon conmon[310082]: 2025-04-08T10:54:30.365+0000
> 7f57d81e0640 -1 cephfs::mirror::Utils connect: error connecting to
bk-site:
> (13) Permission denied
> Apr 08 12:54:30 s1mon cephfs-mirror[310088]: cephfs::mirror::Utils
connect:
> error connecting to bk-site: (13) Permission denied
> Apr 08 12:54:30 s1mon conmon[310082]: 2025-04-08T10:54:30.365+0000
> 7f57d81e0640 -1
> cephfs::mirror::PeerReplayer(f3ea4e15-6d77-4f28-aacb-9afbfe8cc1c5) init:
> error connecting to remote cl>
> Apr 08 12:54:30 s1mon cephfs-mirror[310088]:
> cephfs::mirror::PeerReplayer(f3ea4e15-6d77-4f28-aacb-9afbfe8cc1c5) init:
> error connecting to remote cluster: (13) Permission denied
> Apr 09 00:00:16 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> Apr 09 00:00:16 s1mon conmon[310082]: 2025-04-08T22:00:16.362+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 09 00:00:16 s1mon conmon[310082]: 2025-04-08T22:00:16.386+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 09 00:00:16 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> Apr 09 00:00:16 s1mon conmon[310082]: 2025-04-08T22:00:16.430+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 09 00:00:16 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> Apr 09 00:00:16 s1mon conmon[310082]: 2025-04-08T22:00:16.466+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 09 00:00:16 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> Apr 10 00:00:01 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> Apr 10 00:00:01 s1mon conmon[310082]: 2025-04-09T22:00:01.767+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 10 00:00:01 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> Apr 10 00:00:01 s1mon conmon[310082]: 2025-04-09T22:00:01.811+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 10 00:00:01 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> Apr 10 00:00:01 s1mon conmon[310082]: 2025-04-09T22:00:01.851+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 10 00:00:01 s1mon conmon[310082]: 2025-04-09T22:00:01.891+0000
> 7f57d99e3640 -1 received signal: Hangup from Kernel ( Could be generated
> by pthread_kill(), raise(), abort(), alarm()>
> Apr 10 00:00:01 s1mon cephfs-mirror[310088]: received signal: Hangup
from
> Kernel ( Could be generated by pthread_kill(), raise(), abort(), alarm()
)
> UID: 0
> ----
> Is there any chance I can get the mirroring daemon to forget about the
> cluster I lost ?
>
> Best regards, Jan Zeinstra
> _______________________________________________
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
