[ceph-users] How to size nvme or optane for index pool?

[Szabo, Istvan (Agoda)]

Hi,

How can I know which size of the nvme drive needed for my index pool? At the 
moment I'm using 6x1.92TB NVME (overkill) but I have no idea how is it used.

Thanks


This message is confidential and is for the sole use of the intended 
recipient(s). It may also be privileged or otherwise protected by copyright or 
other legal rules. If you have received it by mistake please let us know by 
reply email and delete it from your system. It is prohibited to copy this 
message or disclose its content to anyone. Any confidentiality or privilege is 
not waived or lost by any mistaken delivery or unauthorized disclosure of the 
message. All messages sent to and from Agoda may be monitored to ensure 
compliance with company policies, to protect the company's interests and to 
remove potential malware. Electronic messages may be intercepted, amended, lost 
or deleted, or contain viruses.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Pool Latency

[Seena Fallah]

Hi,

I'm facing something strange in ceph (v12.2.13, filestore). I have two
clusters with the same config (kernel, network, disks, ...). One of them
has 3ms latency the other has 100ms latency. Both physical disk latency on
write is less than 1ms.
In the cluster with 100ms latency on write when I create another pool with
the same configs (crush rule, replica, ...) and test the latency, it would
like my another cluster. So it seems there is a problem in one of my pools!
The pool has 8212 PGs and each PG is around 12GB with 844 objects. Also, I
have many removed_snaps in this pool and I don't know if it impacts
performance or not?

Do you have any idea what is wrong with my pool? Is there any way to debug
this problem?

Thanks.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] 1U - 16 HDD

[EDH - Manuel Rios]

Hi guys,

I remember some CEPH member deploy CEPH over nodes of 1U and 16 HDD 3,5.

Currently with the chip shortage Supermicro deliver new nodes in October...

Im looking for the model and brand of the 1U and 16 HDD I think is Asus or 
Asrock server nodes... but someone can post the server reference?

Regards
Manuel



___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: "ceph fs perf stats" and "cephfs-top" don't work

[Eugen Block]

Hi,

I just setup a virtual one-node cluster (16.2.5) to check out  
cephfs-top. Regarding the number of clients I was a little surprised,  
too, in the first couple of minutes the number switched back and forth  
between 0 and 1 although I had not connected any client yet. But after  
a while the number got stable and correct. I have two clients  
connected now, but I don't see any stats despite having the stats  
module enabled:


---snip---
cephfs-top - Thu Jul 15 13:35:41 2021
Client(s): 2 - 0 FUSE, 0 kclient, 2 libcephfs

  client_id mount_root chit(%) rlat(s) wlat(s) mlat(s) dlease(%)  
ofiles oicaps oinodes mount_point@host/addr
  24835 /  N/A N/A N/A N/A N/A   N/A   
  N/AN/A N/A@pacific/v1:192.168.124.35
  24846 /clientN/A N/A N/A N/A N/A   N/A   
  N/AN/A N/A@host-192-168-124-168/v1:192.168.124.168

---snip---


The command 'ceph fs perf stats' also only shows this:

---snip---
pacific:~ # ceph fs perf stats
{"version": 1, "global_counters": ["cap_hit", "read_latency",  
"write_latency", "metadata_latency", "dentry_lease", "opened_files",  
"pinned_icaps", "opened_inodes"], "counters": [], "client_metadata":  
{"client.24835": {"IP": "v1:192.168.124.35", "hostname": "pacific",  
"root": "/", "mount_point": "N/A"}, "client.24846": {"IP":  
"v1:192.168.124.168", "hostname": "host-192-168-124-168", "root":  
"/client", "mount_point": "N/A"}}, "global_metrics": {"client.24835":  
[[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]],  
"client.24846": [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0,  
0], [0, 0]]}, "metrics": {"delayed_ranks": [], "mds.0":  
{"client.24835": [], "client.24846": []}}}

---snip---

although I have written a couple of GB into the cephfs.

Regards,
Eugen


Zitat von Erwin Bogaard :


Hi,

I just upgraded our cluster to pacific 16.2.5.
As I'm curious what cephfs-top could give for insights, I followed the
steps in the documentation.
After enabling the mgr module "stats":

# ceph mgr module ls
...
"enabled_modules": [
"dashboard",
"iostat",
"restful",
"stats",
"zabbix"
...

I tried the following command:
# ceph fs perf stats
{"version": 1, "global_counters": ["cap_hit", "read_latency",
"write_latency", "metadata_latency", "dentry_lease", "opened_files",
"pinned_icaps", "opened_inodes"], "counters": [], "client_metadata": {},
"global_metrics": {}, "metrics": {"delayed_ranks": []}}

As you can see, this returns no info whatsoever. The same with:

# cephfs-top
cluster ceph does not exist

The actual cluster name is "ceph".

So I don't understand why "ceph fs perf stats" isn't showing any
information.
Maybe another indicator something isn't ritght:

# ceph fs status
cephfs - 0 clients
==
RANK  STATE  MDSACTIVITY DNSINOS   DIRS   CAPS
...

I see "0 clients". When I take a look in the mgr dashboard, I can actually
see all clients. Which are RHEL 7 & 8 cephfs kernel clients.
There is only 1 mds active, and 1 in standby-replay.
I have multiple pools active, but only 1 fs.

Does anyone have a suggestion where I can take a look enable gathering the
stats?
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io




___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: "ceph fs perf stats" and "cephfs-top" don't work

[Jos Collin]

Contradicting outputs, it seems immediate ones. You need to wait for a 
few seconds for the 'perf stats' (and of-course the cephfs-top) to 
display the correct the metrics. I hope you have ongoing IO while 
running 'perf stats' and cephfs-top. What is your kernel version?


On 15/07/21 5:17 pm, Eugen Block wrote:

Hi,

I just setup a virtual one-node cluster (16.2.5) to check out 
cephfs-top. Regarding the number of clients I was a little surprised, 
too, in the first couple of minutes the number switched back and forth 
between 0 and 1 although I had not connected any client yet. But after 
a while the number got stable and correct. I have two clients 
connected now, but I don't see any stats despite having the stats 
module enabled:


---snip---
cephfs-top - Thu Jul 15 13:35:41 2021
Client(s): 2 - 0 FUSE, 0 kclient, 2 libcephfs

  client_id mount_root chit(%) rlat(s) wlat(s) mlat(s) dlease(%) 
ofiles oicaps oinodes mount_point@host/addr
  24835 /  N/A N/A N/A N/A N/A N/A    
N/A    N/A N/A@pacific/v1:192.168.124.35
  24846 /client    N/A N/A N/A N/A N/A N/A    
N/A    N/A N/A@host-192-168-124-168/v1:192.168.124.168

---snip---


The command 'ceph fs perf stats' also only shows this:

---snip---
pacific:~ # ceph fs perf stats
{"version": 1, "global_counters": ["cap_hit", "read_latency", 
"write_latency", "metadata_latency", "dentry_lease", "opened_files", 
"pinned_icaps", "opened_inodes"], "counters": [], "client_metadata": 
{"client.24835": {"IP": "v1:192.168.124.35", "hostname": "pacific", 
"root": "/", "mount_point": "N/A"}, "client.24846": {"IP": 
"v1:192.168.124.168", "hostname": "host-192-168-124-168", "root": 
"/client", "mount_point": "N/A"}}, "global_metrics": {"client.24835": 
[[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]], 
"client.24846": [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 
0], [0, 0]]}, "metrics": {"delayed_ranks": [], "mds.0": 
{"client.24835": [], "client.24846": []}}}

---snip---

although I have written a couple of GB into the cephfs.

Regards,
Eugen


Zitat von Erwin Bogaard :


Hi,

I just upgraded our cluster to pacific 16.2.5.
As I'm curious what cephfs-top could give for insights, I followed the
steps in the documentation.
After enabling the mgr module "stats":

# ceph mgr module ls
...
    "enabled_modules": [
    "dashboard",
    "iostat",
    "restful",
    "stats",
    "zabbix"
...

I tried the following command:
# ceph fs perf stats
{"version": 1, "global_counters": ["cap_hit", "read_latency",
"write_latency", "metadata_latency", "dentry_lease", "opened_files",
"pinned_icaps", "opened_inodes"], "counters": [], "client_metadata": {},
"global_metrics": {}, "metrics": {"delayed_ranks": []}}

As you can see, this returns no info whatsoever. The same with:

# cephfs-top
cluster ceph does not exist

The actual cluster name is "ceph".

So I don't understand why "ceph fs perf stats" isn't showing any
information.
Maybe another indicator something isn't ritght:

# ceph fs status
cephfs - 0 clients
==
RANK  STATE  MDS    ACTIVITY DNS    INOS   DIRS CAPS
...

I see "0 clients". When I take a look in the mgr dashboard, I can 
actually

see all clients. Which are RHEL 7 & 8 cephfs kernel clients.
There is only 1 mds active, and 1 in standby-replay.
I have multiple pools active, but only 1 fs.

Does anyone have a suggestion where I can take a look enable 
gathering the

stats?
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io




___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io



___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: "ceph fs perf stats" and "cephfs-top" don't work

[Venky Shankar]

On Thu, Jul 15, 2021 at 5:18 PM Eugen Block  wrote:
>
> Hi,
>
> I just setup a virtual one-node cluster (16.2.5) to check out
> cephfs-top. Regarding the number of clients I was a little surprised,
> too, in the first couple of minutes the number switched back and forth
> between 0 and 1 although I had not connected any client yet. But after
> a while the number got stable and correct. I have two clients
> connected now, but I don't see any stats despite having the stats
> module enabled:

The "(dis)appearing" client is the libcephfs instance in mgr/volumes.
When mgr/volumes cleans up its connection, you would see the client
count drop (and increase when it starts instantiating connections).

>
> ---snip---
> cephfs-top - Thu Jul 15 13:35:41 2021
> Client(s): 2 - 0 FUSE, 0 kclient, 2 libcephfs
>
>client_id mount_root chit(%) rlat(s) wlat(s) mlat(s) dlease(%)
> ofiles oicaps oinodes mount_point@host/addr
>24835 /  N/A N/A N/A N/A N/A   N/A
>N/AN/A N/A@pacific/v1:192.168.124.35
>24846 /clientN/A N/A N/A N/A N/A   N/A
>N/AN/A N/A@host-192-168-124-168/v1:192.168.124.168
> ---snip---
>
>
> The command 'ceph fs perf stats' also only shows this:
>
> ---snip---
> pacific:~ # ceph fs perf stats
> {"version": 1, "global_counters": ["cap_hit", "read_latency",
> "write_latency", "metadata_latency", "dentry_lease", "opened_files",
> "pinned_icaps", "opened_inodes"], "counters": [], "client_metadata":
> {"client.24835": {"IP": "v1:192.168.124.35", "hostname": "pacific",
> "root": "/", "mount_point": "N/A"}, "client.24846": {"IP":
> "v1:192.168.124.168", "hostname": "host-192-168-124-168", "root":
> "/client", "mount_point": "N/A"}}, "global_metrics": {"client.24835":
> [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0]],
> "client.24846": [[0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0, 0], [0,
> 0], [0, 0]]}, "metrics": {"delayed_ranks": [], "mds.0":
> {"client.24835": [], "client.24846": []}}}
> ---snip---

The bunch of "N/A"s are due to the client metadata not having entries
for which metrics are valid (sent) by the client. Normally, you should
see something like::

{
  "version": 1,
  "global_counters": [
"cap_hit",
"read_latency",
"write_latency",
"metadata_latency",
"dentry_lease",
"opened_files",
"pinned_icaps",
"opened_inodes"
  ],
  "counters": [],
  "client_metadata": {
"client.624141": {
  "IP": "X.X.X.X",
  "hostname": "host1",
  "root": "/",
  "mount_point": "/mnt/cephfs",
  "valid_metrics": [
"cap_hit",
"read_latency",
"write_latency",
"metadata_latency",
"dentry_lease",
"opened_files",
"pinned_icaps",
"opened_inodes"
  ]
},

What does "ceph tell mds. client ls" dump?

And, as Jos mentioned, it takes a couple of seconds for the stats to
show up when run afresh.

>
> although I have written a couple of GB into the cephfs.
>
> Regards,
> Eugen
>
>
> Zitat von Erwin Bogaard :
>
> > Hi,
> >
> > I just upgraded our cluster to pacific 16.2.5.
> > As I'm curious what cephfs-top could give for insights, I followed the
> > steps in the documentation.
> > After enabling the mgr module "stats":
> >
> > # ceph mgr module ls
> > ...
> > "enabled_modules": [
> > "dashboard",
> > "iostat",
> > "restful",
> > "stats",
> > "zabbix"
> > ...
> >
> > I tried the following command:
> > # ceph fs perf stats
> > {"version": 1, "global_counters": ["cap_hit", "read_latency",
> > "write_latency", "metadata_latency", "dentry_lease", "opened_files",
> > "pinned_icaps", "opened_inodes"], "counters": [], "client_metadata": {},
> > "global_metrics": {}, "metrics": {"delayed_ranks": []}}
> >
> > As you can see, this returns no info whatsoever. The same with:
> >
> > # cephfs-top
> > cluster ceph does not exist
> >
> > The actual cluster name is "ceph".
> >
> > So I don't understand why "ceph fs perf stats" isn't showing any
> > information.
> > Maybe another indicator something isn't ritght:
> >
> > # ceph fs status
> > cephfs - 0 clients
> > ==
> > RANK  STATE  MDSACTIVITY DNSINOS   DIRS   CAPS
> > ...
> >
> > I see "0 clients". When I take a look in the mgr dashboard, I can actually
> > see all clients. Which are RHEL 7 & 8 cephfs kernel clients.
> > There is only 1 mds active, and 1 in standby-replay.
> > I have multiple pools active, but only 1 fs.
> >
> > Does anyone have a suggestion where I can take a look enable gathering the
> > stats?
> > ___
> > ceph-users mailing list -- ceph-users@ceph.io
> > To unsubscribe send an email to ceph-users-le...@ceph.io
>
>
>
> ___
> ceph-users mailing list -- ceph-users@ceph.io
> To unsubscribe send an email to ceph-users-le...@ceph.io
>


-- 
Cheers,
Venky

___
ceph-use

[ceph-users] reset user stats = (75) Value too large for defined data type

[Jean-Sebastien Landry]

Hello ceph-users, does someone have an idea why I got this?

$ radosgw-admin user stats --uid someone --reset-stats
ERROR: could not reset user stats: (75) Value too large for defined data 
type



___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: "ceph fs perf stats" and "cephfs-top" don't work

[Erwin Bogaard]

If I run the commands now, I still don't get any information returned with
ceph fs perf stats.
cephfs-top still says there is no cluster ceph.

The clients are all el7 & el8 kernel clients and show up in the Dashboard.

When I execute "ceph tell mds. client ls", I can see all clients
are there. For example:

[
{
"id": 2916622,
"entity": {
"name": {
"type": "client",
"num": 2916622
},
"addr": {
"type": "v1",
"addr": "10.0.1.24:0",
"nonce": 3286427868
}
},
"state": "open",
"num_leases": 0,
"num_caps": 1,
"request_load_avg": 0,
"uptime": 44154.571553857997,
"requests_in_flight": 0,
"num_completed_requests": 0,
"num_completed_flushes": 0,
"reconnecting": false,
"recall_caps": {
"value": 0,
"halflife": 60
},
"release_caps": {
"value": 0,
"halflife": 60
},
"recall_caps_throttle": {
"value": 0,
"halflife": 1.5
},
"recall_caps_throttle2o": {
"value": 0,
"halflife": 0.5
},
"session_cache_liveness": {
"value": 0,
"halflife": 300
},
"cap_acquisition": {
"value": 0,
"halflife": 10
},
"delegated_inos": [],
"inst": "client.2916622 v1:10.0.1.24:0/3286427868",
"completed_requests": [],
"prealloc_inos": [],
"client_metadata": {
"client_features": {
"feature_bits": "0x7bff"
},
"metric_spec": {
"metric_flags": {
"feature_bits": "0x001f"
}
},
"entity_id": "xxx",
"hostname": "xxx.xxx.xx",
"kernel_version": "4.18.0-305.7.1.el8_4.x86_64",
"root": "/nodes/xxx/xxx"
}
},

So where Eugen gets "some" information back, I don't get any.
It looks as if the cluster isn't recognized, even while all other commands
are working.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Windows Client on 16.2.+

[Robert W. Eckert]

I would like to directly mount cephfs from the windows client, and keep getting 
the error below.


PS C:\Program Files\Ceph\bin> .\ceph-dokan.exe -l x
2021-07-15T17:41:30.365Eastern Daylight Time 4 -1 monclient(hunting): 
handle_auth_bad_method server allowed_methods [2] but i only support [2]
2021-07-15T17:41:30.365Eastern Daylight Time 5 -1 monclient(hunting): 
handle_auth_bad_method server allowed_methods [2] but i only support [2]
2021-07-15T17:41:30.365Eastern Daylight Time 6 -1 monclient(hunting): 
handle_auth_bad_method server allowed_methods [2] but i only support [2]
failed to fetch mon config (--no-mon-config to skip)

My Ceph.conf on the windows client looks like

# minimal ceph.conf for fe3a7cb0-69ca-11eb-8d45-c86000d08867
[global]
fsid = fe3a7cb0-69ca-11eb-8d45-c86000d08867
mon_host = [v2:192.168.2.142:3300/0,v1:192.168.2.142:6789/0] 
[v2:192.168.2.141:3300/0,v1:192.168.2.141:6789/0] 
[v2:192.168.2.199:3300/0,v1:192.168.2.199:6789/0]
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
[client.admin]
keyring = c:/programdata/ceph/ceph.client.admin.keyring

With the same global settings, I can mount ceph on a Ubuntu WSL2 VM, and even 
access it from windows, but would rather have the direct connection.

This appears to be an issue with the ceph client for windows found from 
https://docs.ceph.com/en/latest/install/windows-install/  because the RBD 
command gives a similar error

PS C:\Program Files\Ceph\bin> rbd ls
2021-07-15T17:53:51.244Eastern Daylight Time 5 -1 monclient(hunting): 
handle_auth_bad_method server allowed_methods [2] but i only support [2]
2021-07-15T17:53:51.244Eastern Daylight Time 3 -1 monclient(hunting): 
handle_auth_bad_method server allowed_methods [2] but i only support [2]
2021-07-15T17:53:51.245Eastern Daylight Time 4 -1 monclient(hunting): 
handle_auth_bad_method server allowed_methods [2] but i only support [2]


Is the windows client being maintained?

Thanks,
Rob
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: How to size nvme or optane for index pool?

[Szabo, Istvan (Agoda)]

Yeah, I see the same, 6 servers have nvme drives and today in the iops side 
were all maxed out, but I don’t understand why, the user made head operation 
like 6 sometimes in a minute, the cluster iops was like 60-70k and how can 
this max out 6 nvme drives which should be able to server each more then this 
iops :(

Istvan Szabo
Senior Infrastructure Engineer
---
Agoda Services Co., Ltd.
e: istvan.sz...@agoda.com
---

On 2021. Jul 15., at 13:03, Anthony D'Atri  wrote:

`ceph df `?   Back when I was using SATA SSDs I saw limiting more on iops than 
space.

I’ve seen an estimate of 1% of the size of the buckets pool but this is very 
much a function of your object size distribution.  More but smaller drives 
would decrease your failure domain and speed up recovery.

On Jul 14, 2021, at 9:52 PM, Szabo, Istvan (Agoda)  
wrote:

Hi,

How can I know which size of the nvme drive needed for my index pool? At the 
moment I'm using 6x1.92TB NVME (overkill) but I have no idea how is it used.

Thanks


This message is confidential and is for the sole use of the intended 
recipient(s). It may also be privileged or otherwise protected by copyright or 
other legal rules. If you have received it by mistake please let us know by 
reply email and delete it from your system. It is prohibited to copy this 
message or disclose its content to anyone. Any confidentiality or privilege is 
not waived or lost by any mistaken delivery or unauthorized disclosure of the 
message. All messages sent to and from Agoda may be monitored to ensure 
compliance with company policies, to protect the company's interests and to 
remove potential malware. Electronic messages may be intercepted, amended, lost 
or deleted, or contain viruses.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: How to size nvme or optane for index pool?

[Szabo, Istvan (Agoda)]

So before you have any data, how you can calculate? Like I have 360TB free 
space, for this how much should be the imdex pool.

Istvan Szabo
Senior Infrastructure Engineer
---
Agoda Services Co., Ltd.
e: istvan.sz...@agoda.com
---

On 2021. Jul 15., at 13:24, Konstantin Shalygin  wrote:

What you mean? You can check pool usage via 'ceph df detail' output

Sent from my iPhone

On 15 Jul 2021, at 07:53, Szabo, Istvan (Agoda)  wrote:

How can I know which size of the nvme drive needed for my index pool? At the 
moment I'm using 6x1.92TB NVME (overkill) but I have no idea how is it used.



This message is confidential and is for the sole use of the intended 
recipient(s). It may also be privileged or otherwise protected by copyright or 
other legal rules. If you have received it by mistake please let us know by 
reply email and delete it from your system. It is prohibited to copy this 
message or disclose its content to anyone. Any confidentiality or privilege is 
not waived or lost by any mistaken delivery or unauthorized disclosure of the 
message. All messages sent to and from Agoda may be monitored to ensure 
compliance with company policies, to protect the company's interests and to 
remove potential malware. Electronic messages may be intercepted, amended, lost 
or deleted, or contain viruses.
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Ceph orch terminating mgrs

[Jim Bartlett]

I ran into an issue where I tried to restart my mgrs. With ‘ceph orch restart 
mgr’ and it caused my active manager to go into an endless restart loop. I then 
tried to manually fail over to the other mgr and it began the same restart loop 
behaviour. I then fell into the trap of issuing ‘ceph orch stop mgr’. Both 
managers stopped, thus removing cluster control. When I attempt to start one 
manually with systemctl, it starts fine and begins printing out the cluster 
status line but is terminated within 15 seconds by the orchestrator. I am 
assuming it is the orchestrator because if I pause it, then I can start the 
managers and they run fine. As soon as I resume, it kills both managers. How 
can I stop this orchestrator hell bent on terminating managers?
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: "ceph orch restart mgr" command creates mgr restart loop

[Jim Bartlett]

Did you ever discover the reason for this restart loop?
___
ceph-users mailing list -- ceph-users@ceph.io
To unsubscribe send an email to ceph-users-le...@ceph.io

[ceph-users] Re: "ceph fs perf stats" and "cephfs-top" don't work

[Eugen Block]

Hi,

please find my 'client ls' output at the end.

I was quite patient, I waited for an hour or two, and tried it again  
this morning. I copy a couple of GB into cephfs which takes quite a  
while so there is something like a constant stream of data. This is  
also visible in the daemonperf output, of course, the perf dump also  
shows valid data.


Regards,
Eugen

---snip---
pacific:~ # ceph tell mds.0 client ls
2021-07-16T08:37:21.987+0200 7f9715ffb700  0 client.27311  
ms_handle_reset on v2:192.168.124.35:6800/3365614954
2021-07-16T08:37:22.071+0200 7f9716ffd700  0 client.27313  
ms_handle_reset on v2:192.168.124.35:6800/3365614954

[
{
"id": 24846,
"entity": {
"name": {
"type": "client",
"num": 24846
},
"addr": {
"type": "v1",
"addr": "192.168.124.168:0",
"nonce": 3813501997
}
},
"state": "open",
"num_leases": 0,
"num_caps": 4450,
"request_load_avg": 5130,
"uptime": 68589.6793,
"requests_in_flight": 189,
"num_completed_requests": 0,
"num_completed_flushes": 0,
"reconnecting": false,
"recall_caps": {
"value": 0,
"halflife": 60
},
"release_caps": {
"value": 2203.5658328262321,
"halflife": 60
},
"recall_caps_throttle": {
"value": 0,
"halflife": 1.5
},
"recall_caps_throttle2o": {
"value": 0,
"halflife": 0.5
},
"session_cache_liveness": {
"value": 22304.741127381614,
"halflife": 300
},
"cap_acquisition": {
"value": 0,
"halflife": 10
},
"delegated_inos": [
{
"start": "0x1000455",
"length": 500
}
],
"inst": "client.24846 v1:192.168.124.168:0/3813501997",
"completed_requests": [],
"prealloc_inos": [
{
"start": "0x1000455",
"length": 500
},
{
"start": "0x10151d4",
"length": 809
}
],
"client_metadata": {
"client_features": {
"feature_bits": "0x3bff"
},
"metric_spec": {
"metric_flags": {
"feature_bits": "0x"
}
},
"entity_id": "nova",
"hostname": "host-192-168-124-168",
"kernel_version": "5.3.18-lp152.57-default",
"root": "/client"
}
},
{
"id": 24835,
"entity": {
"name": {
"type": "client",
"num": 24835
},
"addr": {
"type": "v1",
"addr": "192.168.124.35:0",
"nonce": 3089325989
}
},
"state": "open",
"num_leases": 0,
"num_caps": 1,
"request_load_avg": 0,
"uptime": 68774.2717,
"requests_in_flight": 0,
"num_completed_requests": 0,
"num_completed_flushes": 0,
"reconnecting": false,
"recall_caps": {
"value": 0,
"halflife": 60
},
"release_caps": {
"value": 0,
"halflife": 60
},
"recall_caps_throttle": {
"value": 0,
"halflife": 1.5
},
"recall_caps_throttle2o": {
"value": 0,
"halflife": 0.5
},
"session_cache_liveness": {
"value": 0,
"halflife": 300
},
"cap_acquisition": {
"value": 0,
"halflife": 10
},
"delegated_inos": [],
"inst": "client.24835 v1:192.168.124.35:0/3089325989",
"completed_requests": [],
"prealloc_inos": [
{
"start": "0x105",
"length": 499
},
{
"start": "0x100025e",
"length": 501
}
],
"client_metadata": {
"client_features": {
"feature_bits": "0x3bff"
},
"metric_spec": {
"metric_flags": {
"feature_bits": "0x"
}
},
"entity_id": "admin",
"hostname": "pacific",
"kernel_version": "5.3.18-lp152.81-default",
"root": "/"
}
}
]
---snip---


Zitat von Venky Shankar :


On Thu, Jul 15, 2021 at 5:18 PM Eugen Block  wrote:


Hi,

I just setup a virtual one-node cluster (16.2.5) to check out
cephfs-top. Regarding the number of clients I was a little surprised,
too, in the first couple of minutes the number switched ba