Hi all, This most likely boils down to the kernel version of connected clients. We faced the same problem some time ago and just kicked away the client mds connections with a follow up kernel update on the servers doing the mount. I can check the version details later for more clarity.
Best, Laimis J. On Tue, Apr 8, 2025, 16:48 Anthony D'Atri <anthony.da...@gmail.com> wrote: > So, OSD nodes that do not have any OSDs on them yet? > > > On Apr 8, 2025, at 9:41 AM, quag...@bol.com.br wrote: > > > > More complete description: > > > > 1-) I formatted and installed the operating system > > > > 2-) This is "ceph installed": > > > > curl --silent --remote-name --location > https://download.ceph.com/rpm-19.2.1/el9/noarch/cephadm > > chmod +x cephadm > > > > ./cephadm add-repo --release squid > > ./cephadm install > > > > cephadm -v bootstrap --mon-ip 172.27.254.6 --cluster-network > 172.28.254.0/24 --log-to-file > > > > cephadm install ceph-common > > > > > > De: "Anthony D'Atri" <anthony.da...@gmail.com> > > Enviada: 2025/04/08 10:35:22 > > Para: quag...@bol.com.br > > Cc: ebl...@nde.ag, ceph-users@ceph.io > > Assunto: Re: [ceph-users] Ceph squid fresh install > > > > What does “ceph installed” mean? I suspect that this description is not > complete. > > > >> > >> On Apr 8, 2025, at 9:21 AM, quag...@bol.com.br wrote: > >> > >> What is a “storage server”? > >> These are machines that only have the operating system and > ceph installed. > >> > >> > >> > >> De: "Anthony D'Atri" <anthony.da...@gmail.com> > >> Enviada: 2025/04/08 10:19:08 > >> Para: quag...@bol.com.br > >> Cc: ebl...@nde.ag, ceph-users@ceph.io > >> Assunto: Re: [ceph-users] Ceph squid fresh install > >> > >> > >> > >> > On Apr 8, 2025, at 9:13 AM, quag...@bol.com.br wrote: > >> > > >> > These 2 IPs are from the storage servers. > >> > >> What is a “storage server”? > >> > >> > >> > There are no user processes running on them. It only has the > operating system and ceph installed. > >> > >> Nobody said anything about user processes. > >> > >> > > >> > > >> > Rafael. > >> > > >> > De: "Eugen Block" <ebl...@nde.ag> > >> > Enviada: 2025/04/08 09:35:35 > >> > Para: quag...@bol.com.br > >> > Cc: ceph-users@ceph.io > >> > Assunto: Re: [ceph-users] Ceph squid fresh install > >> > > >> > These are your two Luminous clients: > >> > > >> > ---snip--- > >> > { > >> > "name": "unknown.0", > >> > "entity_name": "client.admin", > >> > "addrs": { > >> > "addrvec": [ > >> > { > >> > "type": "none", > >> > "addr": "172.27.254.7:0", > >> > "nonce": 443842330 > >> > } > >> > ] > >> > }, > >> > "socket_addr": { > >> > "type": "none", > >> > "addr": "172.27.254.7:0", > >> > "nonce": 443842330 > >> > }, > >> > "con_type": "client", > >> > "con_features": 3387146417253690110, > >> > "con_features_hex": "2f018fb87aa4aafe", > >> > "con_features_release": "luminous", > >> > ... > >> > > >> > { > >> > "name": "client.104098", > >> > "entity_name": "client.admin", > >> > "addrs": { > >> > "addrvec": [ > >> > { > >> > "type": "v1", > >> > "addr": "172.27.254.6:0", > >> > "nonce": 2027668300 > >> > } > >> > ] > >> > }, > >> > "socket_addr": { > >> > "type": "v1", > >> > "addr": "172.27.254.6:0", > >> > "nonce": 2027668300 > >> > }, > >> > "con_type": "client", > >> > "con_features": 3387146417253690110, > >> > "con_features_hex": "2f018fb87aa4aafe", > >> > "con_features_release": "luminous", > >> > ---snip--- > >> > > >> > Zitat von quag...@bol.com.br: > >> > > >> > > Hi Eugen! Thanks a lot! I was able to find luminous connections, > >> > > but I still can't identify which client process. Here is the output: > >> > > Rafael. > >> > > ────────────────────────────────────────────────────────────────── > >> > > De: "Eugen Block" Enviada: 2025/04/08 04:37:47 Para: > ceph-users@ceph.io > >> > > Assunto: [ceph-users] Re: Ceph squid fresh install Hi, you can query > >> > > the MON sessions to identify your older clients with: ceph tell mon. > >> > > sessions It will show you the IP address, con_features_release > (Luminous) > >> > > and a couple of other things. Zitat von Laura Flores : > Hi Rafael, > >> I > >> > > would not force the min_compat_client to be reef when there are > still > > >> > > luminous clients connected, as it is important for all clients to > be >=Reef > >> > >> to understand/encode the pg_upmap_primary feature in the osdmap. > >> As > >> > > for checking which processes are still luminous, I am copying > @Radoslaw > > >> > > Zarzynski who may be able to help more with that. >> Thanks, > Laura > >> > > Flores >> On Mon, Apr 7, 2025 at 11:30 AM quag...@bol.com.br > > wrote: > >> > >>>> Hi, >> I just did a new Ceph installation and would like to > enable the > >> > > "read >> balancer". >> However, the documentation requires that the > minimum > >> > > client version >> be reef. I checked this information through "ceph > >> > > features" and came across >> the situation of having 2 luminous > clients. >> > >> > > # ceph features >> { >> "mon": [ >> { >> "features": > "0x3f03cffffffdffff", > >> > >>> "release": "squid", >> "num": 2 >> } >> ], >> "mds": [ >> { >> > >> > > "features": "0x3f03cffffffdffff", >> "release": "squid", >> "num": > 2 >> } > >> > >>> ], >> "osd": [ >> { >> "features": "0x3f03cffffffdffff", >> > "release": > >> > > "squid", >> "num": 38 >> } >> ], >> "client": [ >> { >> "features": > >> > > "0x2f018fb87aa4aafe", >> "release": "luminous", >> "num": 2 >> }, > >> { >> > >> > > "features": "0x3f03cffffffdffff", >> "release": "squid", >> "num": > 5 >> } > >> > >>> ], >> "mgr": [ >> { >> "features": "0x3f03cffffffdffff", >> > "release": > >> > > "squid", >> "num": 2 >> } >> ] >> } >>>> I tryed to configure the > minimum > >> > > version to reef and received the >> following alert: >> # ceph osd > >> > > set-require-min-compat-client reef >> Error EPERM: cannot set > >> > > require_min_compat_client to reef: 2 connected >> client(s) look > like > >> > > luminous (missing 0x80000000); add >> --yes-i-really-mean-it to do > it > >> > > anyway >>>> Is it ok do confirm anyway? >> Which processes are > still as > >> > > luminous? >>>> Rafael. >> > _______________________________________________ > >> > >>> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe > send an > >> > > email to ceph-users-le...@ceph.io >>>>>> -- >> Laura Flores >> > She/Her/Hers > >> > >>> Software Engineer, Ceph Storage >>> Chicago, IL >> > lflo...@ibm.com | > >> > > lflo...@redhat.com > M: +17087388804 > > >> > > _______________________________________________ > ceph-users > mailing list > >> > > -- ceph-users@ceph.io > To unsubscribe send an email to > >> > > ceph-users-le...@ceph.io > _______________________________________________ > >> > > ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send > an email > >> > > to ceph-users-le...@ceph.io > >> > > >> > > >> > _______________________________________________ > >> > ceph-users mailing list -- ceph-users@ceph.io > >> > To unsubscribe send an email to ceph-users-le...@ceph.io > >> > > _______________________________________________ > ceph-users mailing list -- ceph-users@ceph.io > To unsubscribe send an email to ceph-users-le...@ceph.io > _______________________________________________ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email to ceph-users-le...@ceph.io
