Hi, I must have followed the wrong instructions (as often seems to be the case when it comes to Ceph). I gave Ceph another try with --fsid, and then it worked.
A missing fsid file cost me an hour, but the OSD subfolder seems to be OK now: mixtile@blade3n1:~$ sudo ls -al /var/lib/ceph/8aad3073-39a1-11f1-bf6e-f2704a1efa 9b/osd.2 total 16 drwxr-xr-x 2 ceph ceph 4096 Jun 1 16:50 . drwx------ 14 167 167 4096 Jun 1 13:47 .. lrwxrwxrwx 1 ceph ceph 93 Jun 1 16:44 block -> /dev/ceph-81bc1761-e8f6-446c- 96f3-eb1b8f92628b/osd-block-9f7fd40d-0698-40b9-8718-62942b03e263 -rw-r--r-- 1 ceph ceph 37 Jun 1 16:50 fsid In the meantime, I've managed to issue this command: sudo ceph-volume lvm activate --all There's one problem though: I get the feedback that ceph-osd@2 is working, but sudo systemctl status ceph-osd@2 says that the daemon failed to start: Jun 01 18:57:43 blade3n1 ceph-osd[317772]: 2026-06-01T18:57:43.403+0200 ffffb584 9040 -1 osd.2 7314 init authentication failed: (13) Permission denied What puzzles me: Why is the OSD daemon started as a normal service and not as a Docker container? Am So., 31. Mai 2026 um 21:27 Uhr schrieb Eugen Block via ceph-users < [email protected]>: > I don't see the '--fsid' flag in your command, it needs to be part of > the bootstrap command. So 'cephadm bootrap --mon-ip IP --fsid FSID' > and so on. > > 4 OSDs istn't that much, it should be doable in a reasonable amount of > time. It took me around two hours to bring my test cluster with 3 OSDs > back up, I think. > > > /var/lib/ceph/osd/ceph-2 > > This is the view from within the cephadm shell, the directory on the > filesystem is : > > /var/lib/ceph/{FSID}/osd.2 > > which is then mapped into the container. > > Zitat von Jacek Rużyczka <[email protected]>: > > >> > >> No, the data is not inaccessible until you wipe the OSDs. > > > > > > Ah, that's good to know that at least the data are still there. > > > > How many OSDs are we talking about here? > > > > > > I've got 4 OSDs, which hold 1.3 TB of data in total. I do have a backup > of > > them on an external HDD, which spit out a handful of warnings during the > > last restore, so destroying the previous SSD, creating new ones, and > trying > > another restore are things I'd like to avoid. > > > > And regarding the fsid, you can specify it in the bootstrap command, > >> you won’t be able to change it afterwards. > > > > > > This is exactly what I tried to do: > > > > sudo cephadm bootstrap --mon-ip $MYIP --config /etc/ceph/ceph.conf~~ > > --allow-overwrite > > > > And /etc/ceph/ceph.conf~~ contains the old FSID: > > > > # minimal ceph.conf for 8aad3073-39a1-11f1-bf6e-f2704a1efa9b > > [global] > > fsid = 8aad3073-39a1-11f1-bf6e-f2704a1efa9b > > mon_host = [v2:10.20.0.11:3300/0,v1:10.20.0.11:6789/0] [v2: > > 10.20.0.12:3300/0,v1:10.20.0.12:6789/0] [v2: > > 10.20.0.13:3300/0,v1:10.20.0.13:6789/0] [v2: > > 10.20.0.14:3300/0,v1:10.20.0.14:6789/0] > > > > Nevertheless, cephadm ignored the old settings and gave the new cluster > a > > generated ID. So I've gotta tear down the cluster and start again? What > > about the key pairs stored in /etc/ceph? > > > > You can simply create the OSD directories and create a symbolic link to > >> the block device. Cephadm simply maps it then. > > > > > > I've now recreated the dir /var/lib/ceph/osd/ceph-2 and have to symlink > > /dev/ceph-81bc1761-e8f6-446c-96f3-eb1b8f92628b/o > > sd-block-9f7fd40d-0698-40b9-8718-62942b03e263 there? Or should ceph-2 be > > the name of the symlink? > > > > I recommend to hire a professional to help you out, just to be on the > safe > >> side. > > > > > > I've got a Ceph expert in our hackspace, but I'm not sure whether he can > > help me. > > > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
