Osds usually try to load the last clean map on boot, but if these osds finished the upgrade it must have hit an assert I think
On Fri, Oct 24, 2025 at 8:06 AM Huseyin Cotuk <[email protected]> wrote: > Hi Can, > > No, I did not add or recreate these OSDs. I tried cephadm to run failing > OSDs with the help of following commands without success: > > cephadm --image quay.io/ceph/ceph:v19 unit --name osd.$i --fsid > 4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f restart > > cephadm --image aade1b12b8e6 run --name osd.$i --fsid > 4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f > > BR, > Huseyin > [email protected] > > > > > > On 24 Oct 2025, at 17:55, Can Özyurt <[email protected]> wrote: > > > > Hi Huseyin, > > > > Did you add or recreate these OSDs after the upgrade? > > > > On Fri, 24 Oct 2025 at 16:42, Huseyin Cotuk <[email protected] <mailto: > [email protected]>> wrote: > >> More debug logs from a failing OSD: > >> > >> Oct 24 17:39:52 ank-backup01 > ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: > 2025-10-24T14:39:52.271+0000 77f47e35c640 20 osd.11 70297 reports for 0 > queries > >> Oct 24 17:39:52 ank-backup01 > ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: > 2025-10-24T14:39:52.271+0000 77f47e35c640 15 osd.11 70297 collect_pg_stats > >> Oct 24 17:39:52 ank-backup01 ceph-osd[4100574]: osd.11 pg_epoch: 70297 > pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] > local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 > les/c/f=69077/68834/39143 sis=70280) > [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 > pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] > PeeringState::prepare_stats_for_publish reporting purged_snaps [] > >> Oct 24 17:39:52 ank-backup01 > ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: > 2025-10-24T14:39:52.271+0000 77f47e35c640 20 osd.11 pg_epoch: 70297 > pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] > local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 > les/c/f=69077/68834/39143 sis=70280) > [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 > pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] > PeeringState::prepare_stats_for_publish reporting purged_snaps [] > >> Oct 24 17:39:52 ank-backup01 ceph-osd[4100574]: osd.11 pg_epoch: 70297 > pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] > local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 > les/c/f=69077/68834/39143 sis=70280) > [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 > pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] > PeeringState::prepare_stats_for_publish publish_stats_to_osd 70297:52872808 > >> Oct 24 17:39:52 ank-backup01 > ceph-4e7e7d1c-22db-49c7-9f24-5a75cd3a3b9f-osd-11[4100566]: > 2025-10-24T14:39:52.271+0000 77f47e35c640 15 osd.11 pg_epoch: 70297 > pg[23.b9s0( v 69083'8392506 (68272'8387884,69083'8392506] > local-lis/les=69076/69077 n=8314181 ec=21807/21807 lis/c=69076/68833 > les/c/f=69077/68834/39143 sis=70280) > [11,NONE,NONE,12,31,21,7,NONE,NONE,26,0]p11(0) r=0 lpr=70297 > pi=[68833,70280)/5 crt=69083'8392506 lcod 0'0 mlcod 0'0 unknown mbc={}] > PeeringState::prepare_stats_for_publish publish_stats_to_osd 70297:52872808 > >> Oct 24 17:39:52 ank-backup01 ceph-osd[4100574]: osd.11 pg_epoch: 70297 > pg[23.5fs2( v 69081'8395094 (68276'8390628,69081'8395094] > local-lis/les=69078/69079 n=8316950 ec=21807/21807 lis/c=69078/68833 > les/c/f=69079/68834/39143 sis=70280) > [NONE,NONE,11,8,14,16,0,20,17,18,NONE]p11(2) r=2 lpr=70297 > pi=[68833,70280)/7 crt=69081'8395094 lcod 0'0 mlcod 0'0 unknown mbc={}] > PeeringState::prepare_stats_for_publish reporting purged_snaps [] > >> > >> > >> Selamlar, > >> Huseyin Cotuk > >> [email protected] <mailto:[email protected]> > >> > >> > >> > >> > >> _______________________________________________ > >> ceph-users mailing list -- [email protected] <mailto: > [email protected]> > >> To unsubscribe send an email to [email protected] <mailto: > [email protected]> > > _______________________________________________ > ceph-users mailing list -- [email protected] > To unsubscribe send an email to [email protected] > _______________________________________________ ceph-users mailing list -- [email protected] To unsubscribe send an email to [email protected]
