[ceph-users] Re: orchestrator behaving strangely

[Holger Naundorf]

Hello,
title should of course be
 "orchestrator behaving strangely"





Regards,
Holger


On 27.06.25 12:26, Eugen Block wrote:

I did give a mgr restart another try (for the last OSD removal - which also did not work - I did already restart the mgr without effect) there is no (immediate- i.e after ~10min) effect now as well - or should I reissue the OSD rm command as well?

Hi,

have you retried it after restarting/failing the mgr?

ceph mgr fail

Quite often this (still) helps.

Zitat von Holger Naundorf <naund...@rz.uni-kiel.de>:

Hello,
we are running a ceph cluster at version:





Problems:

An OSD removal (for disk replacement) gets stuck in the 'purge' step:

ceph orch osd rm 406 --replace

root@aadm01:~# ceph orch osd rm status


(now for more than 24h in this state)






Regards,
Holger

--
Dr. Holger Naundorf
Christian-Albrechts-Universität zu Kiel
Rechenzentrum / HPC / Server und Storage
Tel: +49 431 880-1990
Fax:  +49 431 880-1523
naund...@rz.uni-kiel.de

ceph version 19.2.2 (0eceb0defba60152a8182f7bd87d164b639885b8) squid (stable) and since a few weeks the orchestrator started to misbehave - up to now we could not identify any root cause, so I am fishing in the community to see if there are any hints. OSD  HOST   STATE                    PGS  REPLACE  FORCE  ZAP   DRAIN STARTED AT 406  acn07  done, waiting for purge    0  True     False  True 2025-06-25 09:18:07.650734+00:00 At the same time the orchestrator is not restarting OSD daemons - i.e. an 'ceph orch daemon restart osd.xxx' claims its queuing uo the restart, but it never happens. Other services continue to be controlled correctly via 'ceph orch ...' If anyone has an idea where to poke around or can match this to some known problem - I would appreciate any pointers.

_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

--
Dr. Holger Naundorf
Christian-Albrechts-Universität zu Kiel
Rechenzentrum / HPC / Server und Storage
Tel: +49 431 880-1990
Fax:  +49 431 880-1523
naund...@rz.uni-kiel.de

smime.p7s
Description: S/MIME Cryptographic Signature

_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io