[ceph-users] ceph-osd segmentation fault on arm64 quay.io/ceph/ceph:v18.2.4

2025-01-22 Thread Adam Prycki
Hi, today we attempted to upgrade our ceph cluster from 18.2.0 to 18.2.4. OSDs updated to 18.2.4 couldn't start. I've discovered that ceph-osd in segfaults. It happens always on our setup. I've tested it on ubunut 20.04 with docker and ubuntu 22.04 with podman Example: root@ktsh02:~# podman ru

[ceph-users] Re: Modify or override ceph_default_alerts.yml

2025-01-22 Thread Devin A. Bougie
I just wanted to followup to explain how we ended up with each alert being listed twice, which also prevented our changes to ceph_alerts.yml from taking effect. We only had one prometheus service running, and only one PGImbalance rule in the /var/lib/ceph/{FSID}/prometheus.{host}/etc/prometheu

[ceph-users] Re: osd won't restart

2025-01-22 Thread Dominique Ramaekers
I didn't get any reply on this issue, so I tried some steps: - I removed Apparmor (Ubuntu right...) - I restarted the server - osd's unmanaged: #ceph orch set-unmanaged osd.all-available-devices (because when I want to create lvm's this service was bothering me) - Created lvm on a disk Then on cr

[ceph-users] shell fautly command

2025-01-22 Thread Dominique Ramaekers
Hi, I was removing config keys of osd's that were purged... I accidently entered the following command: #ceph config-key rm osd Feedback "key deleted" :-( Did I do something very stupid??? The output of my osd keys: # ceph config-key ls | grep 'osd' | grep -v history "config/global/osd_po

[ceph-users] Re: shell fautly command

2025-01-22 Thread Frédéric Nass
Hi Dominique, - Le 22 Jan 25, à 15:14, Dominique Ramaekers dominique.ramaek...@cometal.be a écrit : > Hi, > > I was removing config keys of osd's that were purged... > > I accidently entered the following command: #ceph config-key rm osd > > Feedback "key deleted" :-( > > Did I do somet

[ceph-users] Re: Seeking Participation! Take the new Ceph User Stores Survey!

2025-01-22 Thread Devender Singh
Hello Laura Few more suggestions.. 1. As we are facing some issue, can we add more command to control clients using watcher, rbd status pool/image Watchers: watcher=10.160.0.245:0/2076588905 client.12541259 cookie=140446370329088 Some commands to control watcher and kill client.id

[ceph-users] Re: Seeking Participation! Take the new Ceph User Stores Survey!

2025-01-22 Thread Laura Flores
Hi Robin, Getting back to you about how to infer these questions: - How many clients are reading/writing from the Ceph cluster in parallel? > - What is the size of the largest files being stored in your Ceph > cluster(s)? > - What is the size of the largest objects being stored in your Ceph > clu

[ceph-users] Re: osd won't restart

2025-01-22 Thread Dominique Ramaekers
Hi Frédéric, Thanks a lot for your help... I'm getting a bit desperate I keep on having the same issue about permissions (see below) but I've checked. All folders in /var/lib/ceph are owned by ceph:ceph. # ceph orch daemon add osd hvs005:/dev/sde ... /usr/bin/docker: stderr Running command: /us

[ceph-users] Re: Watcher Issue

2025-01-22 Thread Devender Singh
Hello Frederic Thanks for your email. We already verified those and tried killing them and upgrading the k8s and cis-plugin to but nothing helps. Below is the output.. did not report any volume.. # for pod in $(kubectl -n $namespace get pods | grep -E 'rbdplugin|nodeplugin' | grep -v provisi

[ceph-users] Re: Watcher Issue

2025-01-22 Thread Devender Singh
Dear all Sending again. Our mail server was having some issue, not sure if you have received it or not. Exactly Similar issue, https://github.com/ceph/ceph-csi/discussions/4410 Regards Dev > On Jan 21, 2025, at 2:33 PM, Devender Singh wrote: > > Hello Eugen > > Thanks for your reply. >

[ceph-users] Re: osd won't restart

2025-01-22 Thread Frédéric Nass
- Le 22 Jan 25, à 15:31, Dominique Ramaekers dominique.ramaek...@cometal.be a écrit : > I didn't get any reply on this issue, so I tried some steps: > - I removed Apparmor (Ubuntu right...) > - I restarted the server > - osd's unmanaged: #ceph orch set-unmanaged osd.all-available-devices (be

[ceph-users] Re: osd won't restart

2025-01-22 Thread Dominique Ramaekers
Sorry about spamming the mailing list... but I have some additional info. Why didn't I think about it earier, I don’t know; but I just removed the services with "ceph orch daemon rm osd.xx --force" Now my cluster is back in Healty state :) But stil I can't create osd's because of the error belo