I haven't tried it myself yet, but you could bootstrap a fresh cluster
with the flag "--fsid", excerpt from the 'cephadm bootstrap -h' command:
--fsid FSID cluster FSID
and set the FSID your OSDs show. But again, I have no idea if that's
gonna work.
Zitat von Eugen Block <[email protected]>:
You're not gonna be able to reactivate those OSDs because they were
built with a different cluster FSID:
Your OSDs:
cluster fsid 8aad3073-39a1-11f1-bf6e-f2704a1efa9b
Your new cluster:
Inferring fsid 98e04296-5b5a-11f1-84cf-ceccf52b4a0f
And if you have a cephadm-managed cluster, you shouldn't use
ceph-volume directly but via cephadm [0]:
ceph cephadm osd activate <node>
But this won't help you if the cluster's fsid differs as mentioned
above. I think you might be able to recover if you recreate the
monmap by using the OSDs [1]. But this would mean you'd need to tear
down your MONs, it's a difficult situation, I can't tell if it's
gonna work that way. The procedure itself has been verified several
times on this list, but again, it's hard to tell.
[0]
https://docs.ceph.com/en/latest/cephadm/services/osd/#activate-existing-osds
[1]
https://docs.ceph.com/en/latest/rados/troubleshooting/troubleshooting-mon/#recovery-using-osds
Zitat von Jacek Rużyczka via ceph-users <[email protected]>:
Hi,
After being forced to reinstall the O/S on all my Ceph nodes, I'm trying to
reactivate the OSDs, whose data lie on separate drives. The OSDs are
recognised:
mixtile@blade3n3:~$ sudo sudo ceph-volume lvm list
[sudo] password for mixtile:
====== osd.1 =======
[block]
/dev/ceph-5b8338a0-9246-48f5-9cfd-e1bbfa1cc199/osd-block-ccba3dd
2-0503-440f-b7c2-47e8d8e85253
block device
/dev/ceph-5b8338a0-9246-48f5-9cfd-e1bbfa1cc199/o
sd-block-ccba3dd2-0503-440f-b7c2-47e8d8e85253
block uuid 21ftde-FdBB-s4DM-Qeq7-TU4R-vjY7-6VXLoW
cephx lockbox secret
cluster fsid 8aad3073-39a1-11f1-bf6e-f2704a1efa9b
cluster name ceph
crush device class
encrypted 0
osd fsid ccba3dd2-0503-440f-b7c2-47e8d8e85253
osd id 1
osdspec affinity all-available-devices
type block
vdo 0
with tpm 0
devices /dev/nvme0n1
*But: *When trying to link the OSD to the all-new Ceph installation (I've
made several Google searches with contradictory results. This is the last
one.), I get an error:
mixtile@blade3n3:~$ sudo cephadm shell
Inferring fsid 98e04296-5b5a-11f1-84cf-ceccf52b4a0f
Inferring config
/var/lib/ceph/98e04296-5b5a-11f1-84cf-ceccf52b4a0f/mon.blade3n3
/config
Not using image
'sha256:af0c5903e901e329adabe219dfc8d0c3efc1f05102a753902f33ee16
c26b6cee' as it's not in list of non-dangling images with ceph=True label
Unable to find image 'quay.io/ceph/ceph:v19' locally
v19: Pulling from ceph/ceph
Digest:
sha256:af0c5903e901e329adabe219dfc8d0c3efc1f05102a753902f33ee16c26b6cee
Status: Downloaded newer image for quay.io/ceph/ceph:v19
root@blade3n3:/usr/bin# cd
root@blade3n3:~# ceph-volume lvm activate --all
Running command: /usr/bin/ceph-authtool --gen-print-key
Running command: /usr/bin/ceph-authtool --gen-print-key
--> Activating OSD ID 1 FSID ccba3dd2-0503-440f-b7c2-47e8d8e85253
Running command: /usr/bin/mount -t tmpfs tmpfs /var/lib/ceph/osd/ceph-1
Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1
Running command: /usr/bin/ceph-bluestore-tool --cluster=ceph prime-osd-dir
--dev
/dev/ceph-5b8338a0-9246-48f5-9cfd-e1bbfa1cc199/osd-block-ccba3dd2-0503-440f-b7c2-47e8d8e85253
--path /var/lib/ceph/osd/ceph-1 --no-mon-config
stderr: 2026-05-29T17:28:36.854+0000 ffffbd2de040 -1 bdev(0xaaab21623800
/dev/c
eph-5b8338a0-9246-48f5-9cfd-e1bbfa1cc199/osd-block-ccba3dd2-0503-440f-b7c2-47e8d
8e85253) open stat got: (1) Operation not permitted
stderr: failed to read label for
/dev/ceph-5b8338a0-9246-48f5-9cfd-1bbfa1cc199
/osd-block-ccba3dd2-0503-440f-b7c2-47e8d8e85253: (1) Operation not
permitted
--> RuntimeError: command returned non-zero exit status: 1
This puzzles me because I'm already root, but the Bluestore tool won't let
me reactivate the OSD? The old OSD is now full of valuable production data.
And: Isn't it possible to use ceph orch to perform this operation?
Regards
Jacek Rużyczka
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
_______________________________________________
ceph-users mailing list -- [email protected]
To unsubscribe send an email to [email protected]
