[ceph-users] Re: LARGE_OMAP_OBJECTS: any proper action possible?

On Thu, Aug 26, 2021 at 9:49 AM Frank Schilder wrote: > > Hi Dan, > > he he, I built a large omap object cluster, we are up to 5 now :) > > It is possible that our meta-data pool became a bottleneck. I'm re-deploying > OSDs on these disks at the moment, increasing the OSD count from 1 to 4. The

[ceph-users] Re: How to slow down PG recovery when a failed OSD node come back?

Thanks a lot for the tips. I do not have the PG recovery with HDD system, and the issue only happens on SSD. I checked SSD related parameter settings, and did not find anything. Still in pursuit of what could be the root cause... samuel huxia...@horebdata.cn From: Frank Schilder Date: 20

[ceph-users] Re: Not able to reach quorum during update

Hi, as this seems to be an issue only in the specific environment, today i fixed the issue by stopping the remaining octopus mon. After stopping the octopus mon, the both pacific mon's were able to form a quorum. After a manual redeploy (with pacific) of the last octopus mon the issue was fixed

[ceph-users] Ceph as a HDFS alternative?

Hi, all. I want to use ceph instead of HDFS in big data analysis senario, does ceph have some potential problems when the cluster becoming big? say 100PB or 500PB? As far I know, there are some cons: 1. no short circuit read, so we need fast network say 10G or better 50G? 2. not ex

[ceph-users] Re: Ceph as a HDFS alternative?

Ceph cannot scale like HDFS. There are 10K-20K node HDFS clusters in production. There is no data locality concept if you use CEPH, every IO will be served from the network. On Thu, Aug 26, 2021 at 12:04 PM zhang listar wrote: > > Hi, all. > > I want to use ceph instead of HDFS in big data analys

[ceph-users] Re: How to slow down PG recovery when a failed OSD node come back?

For luminous you should check the corresponding _ssd-config values for osd_recovery_sleep and osd_max_backfills. However, I don't think you should see a problem with the defaults with luminous. In fact, I had good experience with making recovery even more aggressive than the defaults. You might

[ceph-users] Re: tcmu-runner crashing on 16.2.5

Thank you for the analysis. Can you think of a workaround for the issue? -Paul Sent from my iPhone On Aug 26, 2021, at 5:17 AM, Xiubo Li wrote:  Hi Paul, There has one racy case when updating the state to ceph cluster and while reopening the image, which will close and open the image, the

[ceph-users] Re: LARGE_OMAP_OBJECTS: any proper action possible?

Hi Dan, he he, I built a large omap object cluster, we are up to 5 now :) It is possible that our meta-data pool became a bottleneck. I'm re-deploying OSDs on these disks at the moment, increasing the OSD count from 1 to 4. The disks I use require high concurrency access to get close to spec pe

[ceph-users] 回复: Re: Ceph as a HDFS alternative?

RBD now support “read_from_replica=localize”, which may reduce some network traffic. But CephFS seems not to support this. But I think it is not easy to tell Hadoop to schedule jobs taking the data location into account. 发件人: Serkan Çoban 发送时间: 2021年8月26日 17:11 收件人:

[ceph-users] Debian 11 Bullseye support

Hi, For this August Debian testing became a Debian stable with LTS support. But I see that only sid repo exists, no testing and no new stable bullseye. May be some one knows, when there are plans to have a bullseye build? Best regards, Arūnas ___ ceph

[ceph-users] Re: Missing OSD in SSD after disk failure

Hi Eugen, once again thank you for your time. One of the servers has 2 OSD (osd.0 and osd.7) osd.0 uses the disk /dev/sdb as the "data" disk and part of the /dev/sdd (nvme) osd.7 uses the disk /dev/sdc as the "data" disk and part of the /dev/sdd (nvme) Regarding block.db /dev/sdc got replaced

[ceph-users] Re: Ceph packages for Rocky Linux

> If rocky8 had a ceph-common I would go with that. rocky/almalinux/centos/rhel 8 don't have a ceph-common package in the base packages. This was true for el7 but not for el8. > It would (presumably) be tested more, since it comes with the original distro. I would avoid that since you won't prob

[ceph-users] Re: Disable autostart of old services

If you're using ceph-volume then you have an extra systemd unit called ceph-volume@lvm-- [1] So you probably want to disable that one too. [1] https://docs.ceph.com/en/latest/ceph-volume/systemd/ Regards, Dimitri On Wed, Aug 25, 2021 at 4:50 AM Marc wrote: > Probably ceph-disk osd's not? che

[ceph-users] Re: tcmu-runner crashing on 16.2.5

This is great. Is there a way to test the fix in my environment? -Paul On Aug 26, 2021, at 11:05 AM, Xiubo Li mailto:xiu...@redhat.com>> wrote: Hi Paul, Ilya, I have fixed it in [1], please help review. Thanks [1] https://github.com/open-iscsi/tcmu-runner/pull/667 On 8/26/21 7:34 PM, Pau

[ceph-users] Re: August Ceph Tech Talk

Hi everyone, We will be starting the Ceph Tech Talk in five minutes: https://bluejeans.com/908675367 On Tue, Aug 24, 2021 at 1:13 PM Mike Perez wrote: > > Hi everyone, > > We have a Ceph Tech Talk scheduled for this Thursday at 17:00 UTC with > Matan Brz on how to use Lua Scripting together with

[ceph-users] Re: [EXTERNAL] Re: OSDs flapping with "_open_alloc loaded 132 GiB in 2930776 extents available 113 GiB"

I'll keep trying to repro and gather diags, but running in containers is making it very hard to run debug commands while the ceph daemons are down. Is this a known problem with a solution? In the meantime, what's the impact of running with the Bitmap Allocator instead of the Hybrid one? I'm ne

[ceph-users] Re: August Ceph Tech Talk

Hi everyone, Here's the recording for this tech talk. Unfortunately, we had some quality issues with the demo, sorry about that: https://www.youtube.com/watch?v=F8zKFl60q9g On Thu, Aug 26, 2021 at 9:55 AM Mike Perez wrote: > > Hi everyone, > > We will be starting the Ceph Tech Talk in five minu

[ceph-users] [errno 13] error connecting to the cluster

Hello, I have an issue about ceph. when i run "ceph -s", i got this error. root#ceph -s [errno 13] error connecting to the cluster and there is key on /etc/ceph, it is same key with ceph-mon -rw-r--r-- 1 root root 137 Mar 12 01:23 ceph.client.admin.keyring -rw-r--r-- 1 root root 65 Mar 12