[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-07 Thread Frédéric Nass
already in, >> I >> hope. >> I do not really understand what commands you are advising us to issue, is >> there >> web page >> that lists a sequence of commands we should run, or please put them in >> e-mail? >> Thank you, --Max >> From: Fréd

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-07 Thread Maxim Goncharov
ommands you are advising us to issue, is there web page that lists a sequence of commands we should run, or please put them in e-mail? Thank you, --Max From: Frédéric Nass Sent: Wednesday, May 7, 2025 6:04 AM To: Milan Kupcevic Cc: ceph-users Subject: [ceph-users

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-07 Thread Frédéric Nass
re > web page > that lists a sequence of commands we should run, or please put them in e-mail? > Thank you, --Max > From: Frédéric Nass > Sent: Wednesday, May 7, 2025 6:04 AM > To: Milan Kupcevic > Cc: ceph-users > Subject: [ceph-users] Re: Need help with MDS stuck

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-07 Thread Maxim Goncharov
From: Frédéric Nass Sent: Wednesday, May 7, 2025 6:04 AM To: Milan Kupcevic Cc: ceph-users Subject: [ceph-users] Re: Need help with MDS stuck in readonly state Hi Milan, Great catch! The fix is in Reef v18.2.5 and has been merged to Squid branch, so it should be in

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-07 Thread Frédéric Nass
Hi Milan, Great catch! The fix is in Reef v18.2.5 and has been merged to Squid branch, so it should be in v19.2.3, I think. No fix for EOL Quincy. @Maxim, there might be a way to recover from this situation (without going through the disaster recovery procedure) by fixing the backtraces or remo

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-06 Thread Milan Kupcevic
Hi Max, This could be related to the reported issue #63259. See: https://tracker.ceph.com/issues/63259#note-11 Milan -- Milan Kupcevic Research Computing Lead Storage Engineer Harvard University HUIT, University Research Computing On 5/1/25 08:22, Maxim Goncharov wrote: Hi Everyone, I hav

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-06 Thread Maxim Goncharov
: Tuesday, May 6, 2025 2:53 AM To: Maxim Goncharov Cc: ceph-users@ceph.io Subject: Re: [ceph-users] Re: Need help with MDS stuck in readonly state Hi, it seems like you only changed the xattr, but that doesn't really move the objects into the new pool, only new objects would be created in the addit

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-05 Thread Eugen Block
Eugen Block Sent: Monday, May 5, 2025 5:47 AM To: ceph-users@ceph.io Subject: [ceph-users] Re: Need help with MDS stuck in readonly state Hi, do you have logs from the MDS? And how exactly did you move the /temp directory to a different pool? That might help understand what happened. There

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-05 Thread Maxim Goncharov
t joinable true), here is a log from MDS, you can see where it puts itself in readonly mode. https://www.cmsaf.mit.edu/various/messages Thankyou, --Max From: Eugen Block Sent: Monday, May 5, 2025 5:47 AM To: ceph-users@ceph.io Subject: [ceph-users] Re:

[ceph-users] Re: Need help with MDS stuck in readonly state

2025-05-05 Thread Eugen Block
Hi, do you have logs from the MDS? And how exactly did you move the /temp directory to a different pool? That might help understand what happened. There was a thread here a couple of weeks ago how about moving contents to a different CephFS pool. Zitat von Maxim Goncharov : Hi Everyone,