[ceph-users] Re: Metrics for object sizes

On 21/04/2021 11:46, Szabo, Istvan (Agoda) wrote: Hi, Is there any clusterwise metric regarding object sizes? I'd like to collect some information about the users what is the object sizes in their buckets. Are you talking about RADOS objects or objects inside RGW buckets? I think you are

[ceph-users] After upgrade to 15.2.11 no access to cluster any more

Hi, I upgraded a test cluster to 15.2.11 and after the upgrade has been finished the cluster status was in HEALTH_WARN because of: AUTH_INSECURE_GLOBAL_ID_RECLAIM_ALLOWED: mons are allowing insecure global_id reclaim According to the documentation at https://docs.ceph.com/en/octopus/rados/operat

[ceph-users] Re: After upgrade to 15.2.11 no access to cluster any more

Hi Robert, That's not very good ! You can probably set that back to false on each mon with: ceph daemon mon.`hostname -s` config set auth_allow_insecure_global_id_reclaim true -- Dan On Thu, Apr 22, 2021 at 9:08 AM Robert Sander wrote: > > Hi, > > I upgraded a test cluster to 15.2.11 and aft

[ceph-users] Re: After upgrade to 15.2.11 no access to cluster any more

Am 22.04.21 um 09:07 schrieb Robert Sander: > What should I do? I should also upgrade the CLI client which still was at 15.2.8 (Ubuntu 20.04) because a "ceph orch upgrade" run only updates the software inside the containers. Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b,

[ceph-users] Download-Mirror eu.ceph.com misses Debian Release file

Hi, to whomever it may concern: The mirror server eu.ceph.com does to carry the Release files for 15.2.11 in https://eu.ceph.com/debian-15.2.11/dists/*/ and 16.2.1 in https://eu.ceph.com/debian-16.2.1/dists/*/ Regards -- Robert Sander Heinlein Consulting GmbH Schwedter Str. 8/9b, 10119 Berlin

[ceph-users] Re: RGW objects has same marker and bucket id in different buckets.

Thanks for the answer. I have 2 question: 1- If I use different user and a bucket what will happen? Is this design only for same user or user independent? 2- If I delete the Source bucket with radosgw-admin or via S3 delete, what will happen to these objects? Also I have more questions if yo

[ceph-users] Re: ceph-csi on openshift

I have been hacking this ceph-csi code to get it to work with mesos. The guys developing this csi ceph driver do not seem to grasp the concept of csi. That it should be a universal driver for OC platforms. They are totally focussing on making ceph work with kubernetes. Recently I noticed that t

[ceph-users] Re: cephadm: how to create more than 1 rgw per host

Does anyone know how to create more than 1 rgw per host? Surely it's not a rare configuration. On 2021/04/19 17:09, i...@z1storage.com wrote: Hi Sebastian, Thank you. Is there a way to create more than 1 rgw per host until this new feature is released? On 2021/04/19 11:39, Sebastian Wagner

[ceph-users] Re: cephadm: how to create more than 1 rgw per host

Hello. Its easy. In ceph.conf copy the rgw fields and change 3 things. 1- name 2- log path name 3- client port. After that feel free to start rgw service with systemctl. Check service status and Tail the rgw log file. Try to read or write and check the logs. If everything works as expected then

[ceph-users] monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]

Hi, I have recently add a new host to ceph and copied /etc/ceph directory to the new host. When I execute the simple ceph command as "ceph -s", get the following error. 021-04-22T14:50:46.226+0300 7ff541141700 -1 monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only supp

[ceph-users] Re: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]

On Thu, Apr 22, 2021 at 3:24 PM Cem Zafer wrote: > > Hi, > I have recently add a new host to ceph and copied /etc/ceph directory to > the new host. When I execute the simple ceph command as "ceph -s", get the > following error. > > 021-04-22T14:50:46.226+0300 7ff541141700 -1 monclient(hunting): >

[ceph-users] rbd snap create now working and just hangs forever

Hi, I have a customer VM that is running fine, but I can not make snapshots anymore. rbd snap create rbd/IMAGE@test-bb-1 just hangs forever. When I checked the status with rbd status rbd/IMAGE it shows one watcher, the cpu node where the VM is running. What can I do to investigate further, witho

[ceph-users] ceph-csi on openshift

Hi, I’m trying to get the ceph-csi working on openshift (I followed this guide: https://docs.ceph.com/en/latest/rbd/rbd-kubernetes/). On openshift it seems you can’t run privileged containers per default and can’t use HostPath etc. For these you need to create a security context constraint (a

[ceph-users] Re: rbd snap create now working and just hangs forever

On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens wrote: > > Hi, > > I have a customer VM that is running fine, but I can not make snapshots > anymore. > rbd snap create rbd/IMAGE@test-bb-1 > just hangs forever. Hi Boris, Run $ rbd snap create rbd/IMAGE@test-bb-1 --debug-ms=1 --debug-rbd=20 let it

[ceph-users] Re: rbd snap create now working and just hangs forever

Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov : > On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens wrote: > > > > Hi, > > > > I have a customer VM that is running fine, but I can not make snapshots > > anymore. > > rbd snap create rbd/IMAGE@test-bb-1 > > just hangs forever. > > Hi Boris, >

[ceph-users] Re: RGW objects has same marker and bucket id in different buckets.

Hi Morphin, On Thu, Apr 22, 2021 at 3:40 AM by morphin wrote: > > Thanks for the answer. > > I have 2 question: > 1- If I use different user and a bucket what will happen? Is this > design only for same user or user independent? It's user independent. > 2- If I delete the Source bucket with ra

[ceph-users] Re: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]

On Thu, Apr 22, 2021 at 5:04 PM Cem Zafer wrote: > > Hi Ilya, > Yes you are correct, I have set auth_allow_insecure_global_id_reclaim to > false. > Host ceph-common package version is 16.2.0 and the cluster ceph -v output is > as follows. > > root@ceph100:~# ceph -v > ceph version 16.2.1 (afb906

[ceph-users] Re: rbd snap create now working and just hangs forever

On Thu, Apr 22, 2021 at 5:08 PM Boris Behrens wrote: > > > > Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov : >> >> On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens wrote: >> > >> > Hi, >> > >> > I have a customer VM that is running fine, but I can not make snapshots >> > anymore. >> > rbd

[ceph-users] Re: rbd snap create now working and just hangs forever

Am Do., 22. Apr. 2021 um 17:27 Uhr schrieb Ilya Dryomov : > On Thu, Apr 22, 2021 at 5:08 PM Boris Behrens wrote: > > > > > > > > Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov < > idryo...@gmail.com>: > >> > >> On Thu, Apr 22, 2021 at 4:20 PM Boris Behrens wrote: > >> > > >> > Hi, > >>

[ceph-users] Re: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]

On Thu, Apr 22, 2021 at 6:01 PM Cem Zafer wrote: > > Thanks Ilya, pointing me out to the right direction. > So if I change the auth_allow_insecure_global_id_reclaim to true means older > userspace clients allowed to the cluster? Yes, but upgrading all clients and setting it to false is recommend

[ceph-users] Re: rbd snap create now working and just hangs forever

On Thu, Apr 22, 2021 at 6:00 PM Boris Behrens wrote: > > > > Am Do., 22. Apr. 2021 um 17:27 Uhr schrieb Ilya Dryomov : >> >> On Thu, Apr 22, 2021 at 5:08 PM Boris Behrens wrote: >> > >> > >> > >> > Am Do., 22. Apr. 2021 um 16:43 Uhr schrieb Ilya Dryomov >> > : >> >> >> >> On Thu, Apr 22, 2021 at

[ceph-users] Re: RGW objects has same marker and bucket id in different buckets.

Its crystal clear now. Thank you so much for the answers. The problem is I cant delete these pending objects with radosgw-admin object rm $obj. Also sync error trim does nothing. Due to this problem I've created a New bucket and copied these objects from the main source (out of cluster) and also c

[ceph-users] Configuring an S3 gateway

I'm trying to configure an S3 gateway with pacific and can't wrap my mind around. In the configuration file, my configuration is: [client.radosgw.fa41] rgw_data = /data/ceph/data/radosgw/$cluster.$id log_file = /data/ceph/logs/$cluster-radosgw.$id.log rgw_frontends = "beast ssl_endpoint=0.

[ceph-users] Zero Reclaim/Trim on RBD image

Hi Friends, We have multiple RBD images/devices. We used fstrim to reclaim space on XFS filesystem on RBD images and it works great. Some of the RBD images, we share as block devices to Vmware using SCST. Any suggestion on how to reclaim space on these devices? The images are about 20TB

[ceph-users] Re: Configuring an S3 gateway

On Thu, Apr 22, 2021 at 2:26 PM Fabrice Bacchella wrote: > > I'm trying to configure an S3 gateway with pacific and can't wrap my mind > around. > > In the configuration file, my configuration is: > > [client.radosgw.fa41] > rgw_data = /data/ceph/data/radosgw/$cluster.$id > log_file = /data/c

[ceph-users] Re: Configuring an S3 gateway

> Le 22 avr. 2021 à 20:40, Casey Bodley a écrit : > > On Thu, Apr 22, 2021 at 2:26 PM Fabrice Bacchella > mailto:fabrice.bacche...@orange.fr>> wrote: >> >> I'm trying to configure an S3 gateway with pacific and can't wrap my mind >> around. >> >> In the configuration file, my configuration i

[ceph-users] Re: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]

On Thu, Apr 22, 2021 at 9:24 PM Cem Zafer wrote: > > Sorry to disturb you again but changing the value to yes doesnt affect > anything. Executing simple ceph command from the client replies the following > error, again. I'm not so sure it is related with that parameter. > Have you any idea what

[ceph-users] Re: ceph-csi on openshift

It does seem you can apply some rbac in openshift: https://docs.openshift.com/container-platform/4.6/authentication/managing-security-context-constraints.html For now I'm having no luck, though I ve used the serviceaccountname now in the custom security context constraint. On 22/04/2021, 12:5

[ceph-users] wrong socket path with ceph daemonperf

I doing a custom, manual installation of ceph. I changed the cluster name from the default one to 'ngceph'. I changed the installation path to /data/ceph. A lot of thing correctly resolves the settings. But one command fails: CLUSTER=/data/ceph/conf/ngceph.conf CLUSTER=ngceph ceph daemonperf cl

[ceph-users] /ceph-osd-prestart.sh does use the configuration

I'm using a rpm build directly from ceph, ceph-osd-16.2.1-0.el8.x86_64 I tweaked the configuration with a custom cluster name and custom installation dir In the unit, there is the configuration: ExecStartPre=/usr/libexec/ceph/ceph-osd-prestart.sh --cluster ${CLUSTER} --id %i And this script che

[ceph-users] Re: wrong socket path with ceph daemonperf

> Le 22 avr. 2021 à 22:13, Fabrice Bacchella a > écrit : > > I doing a custom, manual installation of ceph. I changed the cluster name > from the default one to 'ngceph'. > > I changed the installation path to /data/ceph. > > A lot of thing correctly resolves the settings. But one command f

[ceph-users] Re: Configuring an S3 gateway

I agree. Documentation here is pretty vague. systemd services for osds on ubuntu 20.04 and ceph pacific version 16.2.1 does not work either, so I have to run it manually with /usr/bin/ceph-osd -f --cluster ceph --id some-number --setuser ceph --setgroup ceph I think it would be much better if doc

[ceph-users] Re: monclient(hunting): handle_auth_bad_method server allowed_methods [2] but i only support [2]

On Thu, Apr 22, 2021 at 10:16 PM Cem Zafer wrote: > > This client ceph-common version is 16.2.0, here are the outputs. > > indiana@mars:~$ ceph -v > ceph version 16.2.0 (0c2054e95bcd9b30fdd908a79ac1d8bbc3394442) pacific > (stable) > > indiana@mars:~$ dpkg -l | grep -i ceph-common > ii ceph-commo

[ceph-users] Re: [Suspicious newsletter] Re: cephadm: how to create more than 1 rgw per host

Have you ever tried this? Did it work for you? Istvan Szabo Senior Infrastructure Engineer --- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com --- On 2021. Apr 22., at

[ceph-users] Re: Metrics for object sizes

Objects inside RGW buckets like in couch base software they have their own metrics and has this information. Istvan Szabo Senior Infrastructure Engineer --- Agoda Services Co., Ltd. e: istvan.sz...@agoda.com --

[ceph-users] Re: [Suspicious newsletter] Re: cephadm: how to create more than 1 rgw per host

I don't use cephadm, I'm using the not dockerised deployment, but how you create another one on the same host with this config? This is my RGW section: [client.rgw.xyz-cephmon-2s01.rgw0] host = xyz-cephmon-2s01 keyring = /var/lib/ceph/radosgw/ceph-rgw.xyz-cephmon-2s01.rgw0/keyring log file = /va

[ceph-users] Re: Zero Reclaim/Trim on RBD image

Hi, there's 'rbd sparsify', have you tried that? Zitat von Vikas Rana : Hi Friends, We have multiple RBD images/devices. We used fstrim to reclaim space on XFS filesystem on RBD images and it works great. Some of the RBD images, we share as block devices to Vmware using SCST. Any sugge

[ceph-users] Re: [Suspicious newsletter] Re: cephadm: how to create more than 1 rgw per host

Yes. I tried and used and had no problem. On 5 node i have 2x rgw and sometimes for debuging I add new rgw and delete after the test. Im using Nautilus and never tried on pacific but it should work same way. Try on test Env first. 23 Nis 2021 Cum 04:51 tarihinde Szabo, Istvan (Agoda) < istvan.sz.

[ceph-users] Re: Different ceph versions on nodes in cluster

Hi, you can't really avoid having different versions in a cluster, think about an update/upgrade. Usually it's not an issue, but I would recommend to have that state for as short as possible. Just a recently a customer complained about inconsistent objects on a regular basis although all

[ceph-users] Re: Different ceph versions on nodes in cluster

Den fre 23 apr. 2021 kl 08:45 skrev Eugen Block : > you can't really avoid having different versions in a cluster, think > about an update/upgrade. Usually it's not an issue, but I would > recommend to have that state for as short as possible. Just a recently > a customer complained about inconsist