It's not necessarily a ceph mount that shows older features, but the
kernel version might still be relevant. Just a quick example, a fresh
19.2.1 install on a virtual machine with kernel:
soc9-ceph:~ # uname -r
6.4.0-150600.23.33-default
soc9-ceph:~ # ceph versions
{
"mon": {
"ceph version 19.2.1
(58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)": 1
},
"mgr": {
"ceph version 19.2.1
(58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)": 2
},
"overall": {
"ceph version 19.2.1
(58a7fab8be0a062d730ad7da874972fd3fba59fb) squid (stable)": 3
}
}
Within the cephadm shell, I have only squid features:
[ceph: root@soc9-ceph /]# ceph features
{
"mon": [
{
"features": "0x3f03cffffffdffff",
"release": "squid",
"num": 1
}
],
"client": [
{
"features": "0x3f03cffffffdffff",
"release": "squid",
"num": 4
}
],
"mgr": [
{
"features": "0x3f03cffffffdffff",
"release": "squid",
"num": 2
}
]
}
But if I run the command outside of cephadm shell, I have a Reef feature:
soc9-ceph:~ # ceph features
{
"mon": [
{
"features": "0x3f03cffffffdffff",
"release": "squid",
"num": 1
}
],
"client": [
{
"features": "0x3f01cfbdfffdffff",
"release": "reef",
"num": 1
},
...
The locally installed rpm version is Pacific, though:
soc9-ceph:~ # ceph --version
ceph version 16.2.13-66-g54799ee0666
(54799ee06669271880ee5fc715f99202002aa371) pacific (stable)
If you enforced min-compat-client to reef (ceph osd
set-require-min-compat-client reef --yes-i-really-mean-it), you would
not be able to use Ceph commands from the regular shell of your
system(s), but you would be required to always use the cephadm shell.
Zitat von quag...@bol.com.br:
I added 38 disks.
In short, even though I didn't specify the number of disks, what I want to
say is that it's a clean installation.
The OSD containers are also from the squid version (I didn't configure
anything other than what's in the documentation). I don't understand why
you're asking this kind of question.
Could you tell me all the information you need to help me understand the
problem with the luminous clients?
-------------------------
DE: "Anthony D'Atri" <anthony.da...@gmail.com>
ENVIADA: 2025/04/08 10:44:23
PARA: quag...@bol.com.br
CC: ebl...@nde.ag, ceph-users@ceph.io
ASSUNTO: Re: [ceph-users] Ceph squid fresh install
So, OSD nodes that do not have any OSDs on them yet?
On Apr 8, 2025, at 9:41 AM, quag...@bol.com.br wrote:
More complete description:
1-) I formatted and installed the operating system
2-) This is "ceph installed":
curl --silent --remote-name --location
https://download.ceph.com/rpm-19.2.1/el9/noarch/cephadm
chmod +x cephadm
./cephadm add-repo --release squid
./cephadm install
cephadm -v bootstrap --mon-ip 172.27.254.6 --cluster-network
172.28.254.0/24 --log-to-file
cephadm install ceph-common
-------------------------
DE: "Anthony D'Atri" <anthony.da...@gmail.com>
ENVIADA: 2025/04/08 10:35:22
PARA: quag...@bol.com.br
CC: ebl...@nde.ag, ceph-users@ceph.io
ASSUNTO: Re: [ceph-users] Ceph squid fresh install
What does “ceph installed” mean? I suspect that this description is
not
complete.
On Apr 8, 2025, at 9:21 AM, quag...@bol.com.br wrote:
What is a “storage server”?
These are machines that only have the operating
system and
ceph installed.
-------------------------
DE: "Anthony D'Atri" <anthony.da...@gmail.com>
ENVIADA: 2025/04/08 10:19:08
PARA: quag...@bol.com.br
CC: ebl...@nde.ag, ceph-users@ceph.io
ASSUNTO: Re: [ceph-users] Ceph squid fresh install
On Apr 8, 2025, at 9:13 AM, quag...@bol.com.br wrote:
These 2 IPs are from the storage servers.
What is a “storage server”?
There are no user processes running on them. It only has the operating
system and ceph installed.
Nobody said anything about user processes.
Rafael.
De: "Eugen Block" <ebl...@nde.ag>
Enviada: 2025/04/08 09:35:35
Para: quag...@bol.com.br
Cc: ceph-users@ceph.io
Assunto: Re: [ceph-users] Ceph squid fresh install
These are your two Luminous clients:
---snip---
{
"name": "unknown.0",
"entity_name": "client.admin",
"addrs": {
"addrvec": [
{
"type": "none",
"addr": "172.27.254.7:0",
"nonce": 443842330
}
]
},
"socket_addr": {
"type": "none",
"addr": "172.27.254.7:0",
"nonce": 443842330
},
"con_type": "client",
"con_features": 3387146417253690110,
"con_features_hex": "2f018fb87aa4aafe",
"con_features_release": "luminous",
...
{
"name": "client.104098",
"entity_name": "client.admin",
"addrs": {
"addrvec": [
{
"type": "v1",
"addr": "172.27.254.6:0",
"nonce": 2027668300
}
]
},
"socket_addr": {
"type": "v1",
"addr": "172.27.254.6:0",
"nonce": 2027668300
},
"con_type": "client",
"con_features": 3387146417253690110,
"con_features_hex": "2f018fb87aa4aafe",
"con_features_release": "luminous",
---snip---
Zitat von quag...@bol.com.br:
> Hi Eugen! Thanks a lot! I was able to find luminous connections,
> but I still can't identify which client process. Here is the output:
> Rafael.
>
──────────────────────────────────────────────────────────────────
> De: "Eugen Block" Enviada: 2025/04/08 04:37:47 Para:
ceph-users@ceph.io
> Assunto: [ceph-users] Re: Ceph squid fresh install Hi, you can query
> the MON sessions to identify your older clients with: ceph tell mon.
> sessions It will show you the IP address, con_features_release
(Luminous)
> and a couple of other things. Zitat von Laura Flores : > Hi Rafael,
>> I
> would not force the min_compat_client to be reef when there are
still >
> luminous clients connected, as it is important for all clients to be
>=Reef
>> to understand/encode the pg_upmap_primary feature in the osdmap. >>
As
> for checking which processes are still luminous, I am copying
@Radoslaw >
> Zarzynski who may be able to help more with that. >> Thanks, > Laura
> Flores >> On Mon, Apr 7, 2025 at 11:30 AM quag...@bol.com.br >
wrote:
>>>> Hi, >> I just did a new Ceph installation and would like to
enable the
> "read >> balancer". >> However, the documentation requires that the
minimum
> client version >> be reef. I checked this information through "ceph
> features" and came across >> the situation of having 2 luminous
clients. >>
> # ceph features >> { >> "mon": [ >> { >> "features":
"0x3f03cffffffdffff",
>>> "release": "squid", >> "num": 2 >> } >> ], >> "mds": [ >> { >>
> "features": "0x3f03cffffffdffff", >> "release": "squid", >> "num": 2
>> }
>>> ], >> "osd": [ >> { >> "features": "0x3f03cffffffdffff", >>
"release":
> "squid", >> "num": 38 >> } >> ], >> "client": [ >> { >> "features":
> "0x2f018fb87aa4aafe", >> "release": "luminous", >> "num": 2 >> }, >>
{ >>
> "features": "0x3f03cffffffdffff", >> "release": "squid", >> "num": 5
>> }
>>> ], >> "mgr": [ >> { >> "features": "0x3f03cffffffdffff", >>
"release":
> "squid", >> "num": 2 >> } >> ] >> } >>>> I tryed to configure the
minimum
> version to reef and received the >> following alert: >> # ceph osd
> set-require-min-compat-client reef >> Error EPERM: cannot set
> require_min_compat_client to reef: 2 connected >> client(s) look like
> luminous (missing 0x80000000); add >> --yes-i-really-mean-it to do it
> anyway >>>> Is it ok do confirm anyway? >> Which processes are still
as
> luminous? >>>> Rafael. >>
_______________________________________________
>>> ceph-users mailing list -- ceph-users@ceph.io >> To unsubscribe
send an
> email to ceph-users-le...@ceph.io >>>>>> -- >> Laura Flores >>
She/Her/Hers
>>> Software Engineer, Ceph Storage >>> Chicago, IL >> lflo...@ibm.com
|
> lflo...@redhat.com > M: +17087388804 >
> _______________________________________________ > ceph-users mailing
list
> -- ceph-users@ceph.io > To unsubscribe send an email to
> ceph-users-le...@ceph.io
_______________________________________________
> ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an
email
> to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
_______________________________________________
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
