Thank you so much for the detailed instructions. Here’s logs from the failover to a new node.
Apr 05 20:06:08 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
::ffff:192.168.47.72 - - [05/Apr/2025:20:06:08] "GET /metrics HTTP/1.1" 200 -
"" "Prometheus/2.33.4"
Apr 05 20:06:08 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:08.568+0000 7efcbd800700 0 [prometheus INFO
cherrypy.access.139625005087208] ::ffff:192.168.47.72 - -
[05/Apr/2025:20:06:08] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4"
Apr 05 20:06:10 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
::ffff:192.168.47.71 - - [05/Apr/2025:20:06:10] "GET /metrics HTTP/1.1" 200 -
"" "Prometheus/2.33.4"
Apr 05 20:06:10 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:10.702+0000 7efcb4c00700 0 [prometheus INFO
cherrypy.access.139625005087208] ::ffff:192.168.47.71 - -
[05/Apr/2025:20:06:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4"
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.027+0000 7efd95200700 1 mgr handle_mgr_map
Activating!
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.028+0000 7efd95200700 1 mgr handle_mgr_map I am now
activating
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.040+0000 7efc27e00700 0 [dashboard INFO root]
Stopping engine...
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.040+0000 7efc27e00700 0 [dashboard INFO root]
Stopped engine...
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.041+0000 7efc27e00700 0 [prometheus INFO root]
Stopping engine...
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.041+0000 7efc27e00700 0 [prometheus INFO root]
Stopped engine
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
[05/Apr/2025:20:06:12] ENGINE Bus STOPPING
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.041+0000 7efc43000700 0 [prometheus INFO
cherrypy.error] [05/Apr/2025:20:06:12] ENGINE Bus STOPPING
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.053+0000 7efc3a400700 0 [dashboard INFO root] Engine
stopped.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
[05/Apr/2025:20:06:12] ENGINE HTTP Server
cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.054+0000 7efc43000700 0 [prometheus INFO
cherrypy.error] [05/Apr/2025:20:06:12] ENGINE HTTP Server
cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
[05/Apr/2025:20:06:12] ENGINE Bus STOPPED
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.054+0000 7efc43000700 0 [prometheus INFO
cherrypy.error] [05/Apr/2025:20:06:12] ENGINE Bus STOPPED
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.054+0000 7efc43000700 0 [prometheus INFO root]
Engine stopped.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.056+0000 7efc27e00700 0 [alerts DEBUG root] setting
log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.057+0000 7efc27e00700 0 [alerts INFO root] Init
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.057+0000 7efc27e00700 1 mgr load Constructed class
from module: alerts
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.058+0000 7efc27e00700 0 [balancer DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.058+0000 7efc27e00700 1 mgr load Constructed class
from module: balancer
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.058+0000 7efcfac00700 0 [alerts INFO root] Starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.060+0000 7efc27e00700 0 [cephadm DEBUG root] setting
log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.060+0000 7efcb4c00700 0 [balancer INFO root] Starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.067+0000 7efcb4c00700 0 [balancer INFO root]
Optimize plan auto_2025-04-05_20:06:12
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.067+0000 7efcb4c00700 0 [balancer INFO root] Mode
upmap, max misplaced 0.050000
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.067+0000 7efcb4c00700 0 [balancer INFO root] Some
PGs (1.000000) are unknown; try again later
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.212+0000 7efc27e00700 1 mgr load Constructed class
from module: cephadm
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.213+0000 7efc27e00700 0 [crash DEBUG root] setting
log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.213+0000 7efc27e00700 1 mgr load Constructed class
from module: crash
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.215+0000 7efc27e00700 0 [dashboard DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.215+0000 7efc27e00700 1 mgr load Constructed class
from module: dashboard
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.216+0000 7efc23800700 0 [dashboard INFO
access_control] Loading user roles DB version=2
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.217+0000 7efc23800700 0 [dashboard INFO sso] Loading
SSO DB version=1
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.217+0000 7efc23800700 0 [dashboard INFO root]
server: ssl=yes host=:: port=8443
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.218+0000 7efc27e00700 0 [devicehealth DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.218+0000 7efc27e00700 1 mgr load Constructed class
from module: devicehealth
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.219+0000 7efc27e00700 0 [insights DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.219+0000 7efc27e00700 1 mgr load Constructed class
from module: insights
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.219+0000 7efc20600700 0 [insights INFO root] Reset
curr health slot key health_history/2025-04-05_20 next flush None checks check
count 0
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.219+0000 7efc21a00700 0 [devicehealth INFO root]
Starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.222+0000 7efc27e00700 0 [iostat DEBUG root] setting
log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.222+0000 7efc27e00700 1 mgr load Constructed class
from module: iostat
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.224+0000 7efc27e00700 0 [mirroring DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.227+0000 7efc21a00700 0 [devicehealth INFO root]
Check health
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.235+0000 7efc27e00700 1 mgr load Constructed class
from module: mirroring
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.236+0000 7efc27e00700 0 [nfs DEBUG root] setting log
level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.236+0000 7efc27e00700 1 mgr load Constructed class
from module: nfs
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.237+0000 7efc27e00700 0 [orchestrator DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.237+0000 7efc27e00700 1 mgr load Constructed class
from module: orchestrator
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.239+0000 7efc27e00700 0 [pg_autoscaler DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.239+0000 7efc27e00700 1 mgr load Constructed class
from module: pg_autoscaler
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.239+0000 7efc10200700 0 [pg_autoscaler INFO root]
_maybe_adjust
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.240+0000 7efc27e00700 0 [progress DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.240+0000 7efc27e00700 1 mgr load Constructed class
from module: progress
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.247+0000 7efc27e00700 0 [prometheus DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.248+0000 7efc27e00700 1 mgr load Constructed class
from module: prometheus
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.252+0000 7efc0ee00700 0 [progress INFO root]
Loading...
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.252+0000 7efc27e00700 0 [rbd_support DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.253+0000 7efc0ee00700 0 [progress INFO root] Loaded
[<progress.module.GhostEvent object at 0x7efcf2127710>,
<progress.module.GhostEvent object at 0x7efcf2127cc0>,
<progress.module.GhostEvent object at 0x7efcf2127208>,
<progress.module.GhostEvent object at 0x7efcf2127f28>,
<progress.module.GhostEvent object at 0x7efcf21270f0>,
<progress.module.GhostEvent object at 0x7efcf2127eb8>,
<progress.module.GhostEvent object at 0x7efcf2127b38>,
<progress.module.GhostEvent object at 0x7efcf2127a20>,
<progress.module.GhostEvent object at 0x7efcf2127da0>,
<progress.module.GhostEvent object at 0x7efcf2127f98>,
<progress.module.GhostEvent object at 0x7efcf2127780>,
<progress.module.GhostEvent object at 0x7efcf2127320>,
<progress.module.GhostEvent object at 0x7efcf2127588>,
<progress.module.GhostEvent object at 0x7efcf2127518>,
<progress.module.GhostEvent object at 0x7efcf2127c50>,
<progress.module.GhostEvent object at 0x7efcf21279e8>,
<progress.module.GhostEvent object at 0x7efcf2127d68>,
<progress.module.GhostEvent object at 0x7efcf2127470>,
<progress.module.GhostEvent object at 0x7efcf2127940>,
<progress.module.GhostEvent object at 0x7efcf2127048>,
<progress.module.GhostEvent object at 0x7efcf2127390>,
<progress.module.GhostEvent object at 0x7efcf21276d8>,
<progress.module.GhostEvent object at 0x7efcf2127a90>,
<progress.module.GhostEvent object at 0x7efcf2127ba8>,
<progress.module.GhostEvent object at 0x7efcf2127e48>,
<progress.module.GhostEvent object at 0x7efcf21272e8>,
<progress.module.GhostEvent object at 0x7efcf2127080>,
<progress.module.GhostEvent object at 0x7efcf2127b00>,
<progress.module.GhostEvent object at 0x7efcf21279b0>,
<progress.module.GhostEvent object at 0x7efcf20c1588>,
<progress.module.GhostEvent object at 0x7efcf20c1ac8>,
<progress.module.GhostEvent object at 0x7efcf20c1550>,
<progress.module.GhostEvent object at 0x7efcf20c17f0>,
<progress.module.GhostEvent object at 0x7efcf20c1d68>,
<progress.module.GhostEvent object at 0x7efcf20c1b70>,
<progress.module.GhostEvent object at 0x7efcf20c1c50>,
<progress.module.GhostEvent object at 0x7efcf20c1860>,
<progress.module.GhostEvent object at 0x7efcf20c1908>,
<progress.module.GhostEvent object at 0x7efcf20c14e0>,
<progress.module.GhostEvent object at 0x7efcf20c1cf8>,
<progress.module.GhostEvent object at 0x7efcf20c1470>,
<progress.module.GhostEvent object at 0x7efcf20c1d30>,
<progress.module.GhostEvent object at 0x7efcf20c1e80>,
<progress.module.GhostEvent object at 0x7efcf20c1e10>,
<progress.module.GhostEvent object at 0x7efcf20c1eb8>,
<progress.module.GhostEvent object at 0x7efcf20c1f28>,
<progress.module.GhostEvent object at 0x7efcf20c1668>,
<progress.module.GhostEvent object at 0x7efcf20c16a0>,
<progress.module.GhostEvent object at 0x7efcf20c1ba8>,
<progress.module.GhostEvent object at 0x7efcf20c1c18>] historic events
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.253+0000 7efc0ee00700 0 [progress INFO root] Loaded
OSDMap, ready.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.253+0000 7efc0ee00700 0 [progress INFO root]
Processing OSDMap change 27179..27179
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.256+0000 7efc0d600700 0 [prometheus INFO root]
server_addr: :: server_port: 9283
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.256+0000 7efc0d600700 0 [prometheus INFO root] Cache
enabled
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.256+0000 7efc08600700 0 [prometheus INFO root]
starting metric collection thread
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.258+0000 7efc0d600700 0 [prometheus INFO root]
Starting engine...
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
[05/Apr/2025:20:06:12] ENGINE Bus STARTING
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.258+0000 7efc0d600700 0 [prometheus INFO
cherrypy.error] [05/Apr/2025:20:06:12] ENGINE Bus STARTING
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.259+0000 7efc07200700 0 [rbd_support INFO root]
recovery thread starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.259+0000 7efc07200700 0 [rbd_support INFO root]
starting setup
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.259+0000 7efc27e00700 1 mgr load Constructed class
from module: rbd_support
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.301+0000 7efc27e00700 0 [restful DEBUG root] setting
log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.301+0000 7efc27e00700 1 mgr load Constructed class
from module: restful
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.303+0000 7efc07200700 0 [rbd_support INFO root]
MirrorSnapshotScheduleHandler: load_schedules
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.305+0000 7efc07200700 0 [rbd_support INFO root]
load_schedules: csxyz, start_after=
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.307+0000 7efc04a00700 0 [restful INFO root]
server_addr: :: server_port: 8003
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.310+0000 7efc27e00700 0 [snap_schedule DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.312+0000 7efc04a00700 0 [restful WARNING root]
server not running: no certificate configured
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
Warning: Permanently added '192.168.47.73' (ECDSA) to the list of known hosts.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.319+0000 7efc23800700 0 [dashboard INFO root]
Configured CherryPy, starting engine...
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
Warning: Permanently added '192.168.47.72' (ECDSA) to the list of known hosts.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
Warning: Permanently added '192.168.47.71' (ECDSA) to the list of known hosts.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.361+0000 7efbf8200700 0 [rbd_support INFO root]
MirrorSnapshotScheduleHandler: starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.369+0000 7efbf7800700 0 [rbd_support INFO root]
PerfHandler: starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.371+0000 7efc07200700 0 [rbd_support INFO root]
load_task_task: csxyz, start_after=
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.389+0000 7efc27e00700 1 mgr load Constructed class
from module: snap_schedule
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.390+0000 7efc27e00700 0 [status DEBUG root] setting
log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.390+0000 7efc27e00700 1 mgr load Constructed class
from module: status
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.391+0000 7efc27e00700 0 [telemetry DEBUG root]
setting log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.392+0000 7efc27e00700 1 mgr load Constructed class
from module: telemetry
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.393+0000 7efc27e00700 0 [volumes DEBUG root] setting
log level based on debug_mgr: INFO (2/5)
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.409+0000 7efc27e00700 0 [volumes INFO
volumes.fs.async_job] queuing job for volume 'cssnfsxyz'
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.410+0000 7efc27e00700 0 [volumes INFO
volumes.fs.async_job] queuing job for volume 'cssnfsxyz'
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.410+0000 7efc27e00700 1 mgr load Constructed class
from module: volumes
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
CherryPy Checker:
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
The Application mounted at '' has an empty config.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
[05/Apr/2025:20:06:12] ENGINE Serving on http://:::9283
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.413+0000 7efc0d600700 0 [prometheus INFO
cherrypy.error] [05/Apr/2025:20:06:12] ENGINE Serving on http://:::9283
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
[05/Apr/2025:20:06:12] ENGINE Bus STARTED
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.413+0000 7efc0d600700 0 [prometheus INFO
cherrypy.error] [05/Apr/2025:20:06:12] ENGINE Bus STARTED
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.413+0000 7efc0d600700 0 [prometheus INFO root]
Engine started.
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.416+0000 7efbec400700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.416+0000 7efbec400700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.416+0000 7efbec400700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.416+0000 7efbec400700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.416+0000 7efbec400700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.417+0000 7efbe9200700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.417+0000 7efbe9200700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.417+0000 7efbe9200700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.417+0000 7efbe9200700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.417+0000 7efbe9200700 -1 client.0 error registering
admin socket command: (17) File exists
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.431+0000 7efbd8400700 0 [rbd_support INFO root]
TaskHandler: starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.432+0000 7efc07200700 0 [rbd_support INFO root]
TrashPurgeScheduleHandler: load_schedules
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.434+0000 7efc07200700 0 [rbd_support INFO root]
load_schedules: csxyz, start_after=
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.445+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: CephFS ->
/api/cephfs
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.448+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: CephFsUi ->
/ui-api/cephfs
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.450+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: IscsiUi ->
/ui-api/iscsi
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.450+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Iscsi ->
/api/iscsi
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.450+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: IscsiTarget ->
/api/iscsi/target
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.451+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
NFSGaneshaCluster -> /api/nfs-ganesha/cluster
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.452+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
NFSGaneshaExports -> /api/nfs-ganesha/export
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.453+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: NFSGaneshaUi
-> /ui-api/nfs-ganesha
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.453+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Rbd ->
/api/block/image
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.455+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RbdStatus ->
/ui-api/block/rbd
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.455+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RbdSnapshot ->
/api/block/image/{image_spec}/snap
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.456+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RbdTrash ->
/api/block/image/trash
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.457+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RbdNamespace
-> /api/block/pool/{pool_name}/namespace
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.458+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Pool ->
/api/pool
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.460+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: PoolUi ->
/ui-api/pool
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.461+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RBDPool ->
/api/pool
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.462+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Orchestrator
-> /ui-api/orchestrator
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.462+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Service ->
/api/service
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.463+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RbdMirroring
-> /api/block/mirroring
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.463+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
RbdMirroringSummary -> /api/block/mirroring/summary
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.464+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
RbdMirroringPoolMode -> /api/block/mirroring/pool
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.464+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
RbdMirroringPoolBootstrap -> /api/block/mirroring/pool/{pool_name}/bootstrap
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.464+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
RbdMirroringPoolPeer -> /api/block/mirroring/pool/{pool_name}/peer
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.465+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
RbdMirroringStatus -> /ui-api/block/mirroring
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.466+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Rgw ->
/ui-api/rgw
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.466+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RgwDaemon ->
/api/rgw/daemon
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.466+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RgwSite ->
/api/rgw/site
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.467+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RgwBucket ->
/api/rgw/bucket
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.468+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RgwUser ->
/api/rgw/user
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.472+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Auth ->
/api/auth
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.472+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Cluster ->
/api/cluster
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.473+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
ClusterConfiguration -> /api/cluster_conf
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.474+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: CrushRule ->
/api/crush_rule
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.476+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: CrushRuleUi ->
/ui-api/crush_rule
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.476+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Daemon ->
/api/daemon
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.477+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Docs -> /docs
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.477+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
ErasureCodeProfile -> /api/erasure_code_profile
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.478+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
ErasureCodeProfileUi -> /ui-api/erasure_code_profile
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.478+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
FrontendLogging -> /ui-api/logging
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.478+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Grafana ->
/api/grafana
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.479+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Host ->
/api/host
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.480+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: HostUi ->
/ui-api/host
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.481+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Health ->
/api/health
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.481+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: HomeController
-> /
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.481+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
LangsController -> /ui-api/langs
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.482+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
LoginController -> /ui-api/login
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.482+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Logs ->
/api/logs
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.482+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: MgrModules ->
/api/mgr/module
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.483+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Monitor ->
/api/monitor
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.484+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Osd -> /api/osd
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.486+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: OsdUi ->
/ui-api/osd
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.487+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
OsdFlagsController -> /api/osd/flags
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.488+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: MdsPerfCounter
-> /api/perf_counters/mds
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.488+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: MonPerfCounter
-> /api/perf_counters/mon
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.489+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: OsdPerfCounter
-> /api/perf_counters/osd
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.489+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: RgwPerfCounter
-> /api/perf_counters/rgw
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.490+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
RbdMirrorPerfCounter -> /api/perf_counters/rbd-mirror
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.490+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: MgrPerfCounter
-> /api/perf_counters/mgr
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.491+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
TcmuRunnerPerfCounter -> /api/perf_counters/tcmu-runner
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.491+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: PerfCounters
-> /api/perf_counters
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.491+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
PrometheusReceiver -> /api/prometheus_receiver
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.492+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Prometheus ->
/api/prometheus
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.493+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
PrometheusNotifications -> /api/prometheus/notifications
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.493+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Role ->
/api/role
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.494+0000 7efbd7a00700 0 [rbd_support INFO root]
TrashPurgeScheduleHandler: starting
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.495+0000 7efc07200700 0 [rbd_support INFO root]
setup complete
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.495+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Scope ->
/ui-api/scope
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.496+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Saml2 ->
/auth/saml2
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.496+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Settings ->
/api/settings
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.497+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
StandardSettings -> /ui-api/standard_settings
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.498+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Summary ->
/api/summary
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.498+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Task ->
/api/task
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.498+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: Telemetry ->
/api/telemetry
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.499+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller: User ->
/api/user
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.500+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
UserPasswordPolicy -> /api/user
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.500+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
UserChangePassword -> /api/user/{username}
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.501+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
FeatureTogglesEndpoint -> /api/feature_toggles
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.501+0000 7efc23800700 0 [dashboard INFO
dashboard.controllers._base_controller] Initializing controller:
MessageOfTheDay -> /ui-api/motd
Apr 05 20:06:12 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:12.710+0000 7efc23800700 0 [dashboard INFO
dashboard.module] Engine started.
Apr 05 20:06:13 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:13.039+0000 7efc27400700 0 log_channel(cluster) log
[DBG] : pgmap v3: 609 pgs: 609 active+clean; 1.1 TiB data, 3.3 TiB used, 106
TiB / 109 TiB avail
Apr 05 20:06:13 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
::ffff:192.168.47.73 - - [05/Apr/2025:20:06:13] "GET /metrics HTTP/1.1" 200
39714 "" "Prometheus/2.33.4"
Apr 05 20:06:13 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:13.287+0000 7efc02200700 0 [prometheus INFO
cherrypy.access.139624858224400] ::ffff:192.168.47.73 - -
[05/Apr/2025:20:06:13] "GET /metrics HTTP/1.1" 200 39714 "" "Prometheus/2.33.4"
Apr 05 20:06:14 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:14.037+0000 7efc26000700 0 log_channel(cluster) log
[DBG] : pgmap v4: 609 pgs: 609 active+clean; 1.1 TiB data, 3.3 TiB used, 106
TiB / 109 TiB avail
Apr 05 20:06:16 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:16.038+0000 7efc26000700 0 log_channel(cluster) log
[DBG] : pgmap v5: 609 pgs: 609 active+clean; 1.1 TiB data, 3.3 TiB used, 106
TiB / 109 TiB avail
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.106+0000 7efc1fc00700 0 [insights INFO root]
Received health check update 0 pending
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.149+0000 7efc20600700 0 [insights INFO root]
Applying 1 health updates to slot key health_history/2025-04-05_20 next flush
None checks check count 0
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.149+0000 7efc20600700 0 [insights INFO root] Maybe
flushing slot key health_history/2025-04-05_20 next flush None checks check
count 0 needed False
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.256+0000 7efc0ee00700 0 [progress INFO root]
Processing OSDMap change 27179..27179
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.518+0000 7efce9400700 0 [cephadm INFO root] Detected
new or changed devices on cn01.ceph.xyz.corp
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.518+0000 7efce9400700 0 log_channel(cephadm) log
[INF] : Detected new or changed devices on cn01.ceph.xyz.corp
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.611+0000 7efcd7c00700 0 [cephadm INFO root] Detected
new or changed devices on cn03.ceph.xyz.corp
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.611+0000 7efcd7c00700 0 log_channel(cephadm) log
[INF] : Detected new or changed devices on cn03.ceph.xyz.corp
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.726+0000 7efccf000700 0 [cephadm INFO root] Detected
new or changed devices on cn02.ceph.xyz.corp
Apr 05 20:06:17 cn02.ceph.xyz.corp
ceph-95f49c1c-b1e8-11ee-b5d0-0cc47a8f35c1-mgr-cn02-ceph-xyz-corp-ggixgj[2357414]:
debug 2025-04-05T20:06:17.726+0000 7efccf000700 0 log_channel(cephadm) log
[INF] : Detected new or changed devices on cn02.ceph.xyz.corp
> On Saturday, Apr 05, 2025 at 4:51 AM, Eugen Block <ebl...@nde.ag
> (mailto:ebl...@nde.ag)> wrote:
> Please don't drop the list from your response.
>
> The orchestrator might be down, but you should have an active mgr
> (ceph -s). On the host running the mgr, you can just run 'cephadm logs
> --name mgr.<your_mgr>', or 'journalctl -u
> ceph-<ceph_fsid>@mgr.<your_mgr>'.
> To get fresh logs, you can just run 'ceph mgr fail', look on which
> host a different mgr takes over and gather logs from there.
>
> Zitat von Jeremy Hansen <jer...@skidrow.la>:
>
> > Thanks for the clarification on version jumps. I interpreted the
> > docs to mean that I had to incrementally step through each release.
> > This will save a lot of time.
> >
> > Since the orchestrator is down and I’m not sure how to trigger a
> > failure, I don’t know how to gather those logs. I apologize. I
> > appreciate your assistance.
> >
> > -jeremy
> >
> > > On Saturday, Apr 05, 2025 at 4:01 AM, Eugen Block <ebl...@nde.ag
> > > (mailto:ebl...@nde.ag)> wrote:
> > > It might be a different issue, can you paste the entire stack trace
> > > from the mgr when it's failing?
> > >
> > > Also, you could go directly from Pacific to Reef, there's no need to
> > > upgrade to Quincy (which is EOL). And I would also recommend to
> > > upgrade to a latest major version, e.g. 17.2.8, not .0, otherwise you
> > > might hit bugs which have already been fixed in later releases.
> > >
> > > Zitat von Jeremy Hansen <jer...@skidrow.la>:
> > >
> > > > I ran in to the “Error ENOENT: Module not found” issue with the
> > > > orchestrator. I see the note in the cephadm upgrade docs but I don’t
> > > > quite know what action to take to fix this:
> > > >
> > > > ceph versions
> > > > {
> > > > "mon": {
> > > > "ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516)
> > > > pacific (stable)": 3
> > > > },
> > > > "mgr": {
> > > > "ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516)
> > > > pacific (stable)": 1,
> > > > "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e)
> > > > quincy (stable)": 2
> > > > },
> > > > "osd": {
> > > > "ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516)
> > > > pacific (stable)": 15
> > > > },
> > > > "mds": {
> > > > "ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516)
> > > > pacific (stable)": 3
> > > > },
> > > > "overall": {
> > > > "ceph version 16.2.15 (618f440892089921c3e944a991122ddc44e60516)
> > > > pacific (stable)": 22,
> > > > "ceph version 17.2.0 (43e2e60a7559d3f46c9d53f1ca875fd499a1e35e)
> > > > quincy (stable)": 2
> > > > }
> > > > }
> > > >
> > > > ceph orch upgrade status
> > > > Error ENOENT: Module not found
> > > >
> > > > ceph config-key get mgr/cephadm/osd_remove_queue
> > > > []
> > > >
> > > > Thanks. I’m trying to upgrade this cluster all the way to current.
> > > >
> > > > -jeremy
> > >
> > >
> > > _______________________________________________
> > > ceph-users mailing list -- ceph-users@ceph.io
> > > To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
>
signature.asc
Description: PGP signature
_______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
