[ceph-users] Re: Problems after update Debian bullseye to trixie and quincy to reef

Matthew Darwin Fri, 22 Aug 2025 04:58:24 -0700

Please follow this tracker: https://tracker.ceph.com/issues/64213


(basically dashboard does not work on reef/debian)

On 2025-08-22 07:26, i...@hcpeters.de wrote:

Hi EUgen

great, that sounds not too bad.

Here is the output:
ceph health detail

HEALTH_WARN 15 mgr modules have failed dependencies
[WRN] MGR_MODULE_DEPENDENCY: 15 mgr modules have failed dependencies
     Module 'balancer' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'crash' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'dashboard' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'devicehealth' has failed dependency: PyO3 modules do not yet 
support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'iostat' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'nfs' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'orchestrator' has failed dependency: PyO3 modules do not yet 
support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'pg_autoscaler' has failed dependency: PyO3 modules do not yet 
support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'progress' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'prometheus' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'rbd_support' has failed dependency: PyO3 modules do not yet 
support subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'restful' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'status' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'telemetry' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576
     Module 'volumes' has failed dependency: PyO3 modules do not yet support 
subinterpreters, seehttps://github.com/PyO3/pyo3/issues/576

Regards,

Christian

Eugen Block<ebl...@nde.ag> hat am 22.08.2025 12:49 CEST geschrieben:

Hi,


this is absolutely the right place to ask.

First, I would separate the issues (dashboard and crash). And there's
no need to redeploy anything, it's just a mgr issue (and crash),
nothing critical regarding the operational state of your cluster.

Can you paste the output of 'ceph health detail'? And then you should
look into the active mgr log and search for stack traces or errors in
general.

I would ignore crash for now, fix the mgr first.

Regards,
Eugen

Zitat voni...@hcpeters.de:

Hi all,

not sure it's ok to ask questions here as we are only a little
Theater where I manage the data of our sound and video files.
I set up a ceph cluster as second backup location on bullseye with
quincy which worked flawless.
All runs smooth but I thought I would have to do a step forward to
get on track.
Yes I know: never change a running system. :-D
After updating all three nodes I forget to set the ceph packages on
hold so they were updated to reef with trixie. No way to downgrade
...I tried a lot....! So I went forward.
After all all 3 nodes are back online running reef my cluster seems
to work, my cephfs share is reachable by my client machine, backups
are working.
This is my ceph status output:

cluster:
id: c5986871-ab9f-4365-8113-a1854fa07786
health: HEALTH_WARN
15 mgr modules have failed dependencies

services:
mon: 3 daemons, quorum ceph1,ceph2,ceph3 (age 19h)
mgr: ceph1(active, since 18h), standbys: ceph2, ceph3
mds: 1/1 daemons up, 2 standby
osd: 24 osds: 24 up (since 19h), 24 in (since 22M)

data:
volumes: 1/1 healthy
pools: 3 pools, 1041 pgs
objects: 3.34M objects, 11 TiB
usage: 32 TiB used, 56 TiB / 87 TiB avail
pgs: 1041 active+clean

Problem: dashboard is not working any more and I have this erre/warn:

15 mgr modules have failed dependencies


The cluster was made without podman/containers.

ceph-crash gives me a flood of this errors:

...

WARNING:ceph-crash:post/var/lib/ceph/crash/2025-08-21T11:51:45.411885Z_738ee242-f7bd-4f6c-bd90-d96a08e06bf6 as client.admin failed: 2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or

directory
2025-08-22T06:55:50.234+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff728064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff728068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.238+0000 7ff72cea96c0 -1
AuthRegistry(0x7ff72cea8410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)

WARNING:ceph-crash:post/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash.ceph1 failed: 2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or

directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81ac0640c8) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81ac0686c0) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.318+0000 7f81b3ea96c0 -1
AuthRegistry(0x7f81b3ea8410) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)

WARNING:ceph-crash:post/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.crash failed: 2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or

directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5b9c064068) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5b9c068540) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.398+0000 7f5ba2ea96c0 -1
AuthRegistry(0x7f5ba2ea8410) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)

WARNING:ceph-crash:post/var/lib/ceph/crash/2025-08-21T10:54:08.817091Z_34eaeca9-7883-4b93-b75d-83102c113baa as client.admin failed: 2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or

directory
2025-08-22T06:55:50.478+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc844064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc844068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.482+0000 7fc8494fe6c0 -1
AuthRegistry(0x7fc8494fd410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)

WARNING:ceph-crash:post/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash.ceph1 failed: 2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or

directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d40640c8) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d40686c0) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.562+0000 7f83d9aa96c0 -1
AuthRegistry(0x7f83d9aa8410) no keyring found at
/etc/ceph/ceph.client.crash.ceph1.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)

WARNING:ceph-crash:post/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.crash failed: 2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or

directory
2025-08-22T06:55:50.642+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a30064068) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a30068540) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.646+0000 7f2a37aa96c0 -1
AuthRegistry(0x7f2a37aa8410) no keyring found at
/etc/ceph/ceph.client.crash.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)

WARNING:ceph-crash:post/var/lib/ceph/crash/2025-08-21T10:28:14.730789Z_ad2ef583-85b7-4cf0-a2fd-2fa3b775a7a2 as client.admin failed: 2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a keyring on /etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin: (2) No such file or

directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa800064068) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa800068540) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1 auth: unable to find a
keyring on
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin:
 (2) No such file or
directory
2025-08-22T06:55:50.726+0000 7fa805ea96c0 -1
AuthRegistry(0x7fa805ea8410) no keyring found at
/etc/ceph/ceph.client.admin.keyring,/etc/ceph/ceph.keyring,/etc/ceph/keyring,/etc/ceph/keyring.bin,
 disabling
cephx
[errno 2] RADOS object not found (error connecting to the cluster)
......

In /etc/ceph I have

ceph.client.admin.keyring
ceph.conf
rbdmap


Maye there is a way to fix this but if it's to complicated I will
think about a complete re-install.
I'm not an admin but an amateur with some linux knowledge.
If there is hope and someone could support to solev this I would be
very happy!

Regards,

Christian Peters



_______________________________________________
ceph-users mailing list --ceph-users@ceph.io
To unsubscribe send an email toceph-users-le...@ceph.io


_______________________________________________
ceph-users mailing list --ceph-users@ceph.io
To unsubscribe send an email toceph-users-le...@ceph.io

_______________________________________________
ceph-users mailing list --ceph-users@ceph.io
To unsubscribe send an email toceph-users-le...@ceph.io

_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: Problems after update Debian bullseye to trixie and quincy to reef

Reply via email to