Re: [ceph-users] ceph mimic and samba vfs_ceph

On 08.05.19 23:23, Gregory Farnum wrote: > Fixing the wiring wouldn't be that complicated if you can hack on the > code at all, but there are some other issues with the Samba VFS > implementation that have prevented anyone from prioritizing it so far. > (Namely, smb forks for every incoming client

[ceph-users] 'ceph features' showing wrong releases after upgrade to nautilus?

Hi, I've upgraded our ceph cluster from luminous to nautilus yesterday. There was a little hickup after activating msgr2, but everything else went well without any problem. But the upgrade is not reflected by the output of 'ceph features' (yet?): # ceph --version ceph version 14.2.1 (d555a

[ceph-users] Combining balancer and pg auto scaler?

Hi, with the upgrade to nautilus I was finally able to adjust the PG number for our pools. This process is still running, One pool is going to grow from 256 to 1024 PG since its content has grown significantly over the last month. As a result of the current imbalance, the OSDs ' used capacit

Re: [ceph-users] OSDs failing to boot

Hi Paul, could you please set both "debug bluestore" and "debug bluefs" to 20, run again and share the resulting log. Thanks, Igor On 5/9/2019 2:34 AM, Rawson, Paul L. wrote: Hi Folks, I'm having trouble getting some of my OSDs to boot. At some point, these disks got very full. I fixed the

[ceph-users] Is there a Ceph-mon data size partition max limit?

Hi folks, What is the commanded size for the ceph-mon data partitions? Is there a maximum limit to it? If not is there a way to limit it's growth (or celan it up)? To my knowledge ceph-mon doesn't use a lot of data (500MB - 1GB should be enough, but I'm not the expert here :) We are working on

Re: [ceph-users] Clients failing to respond to cache pressure

Thanks for the info Patrick. We are using ceph packages from ubuntu main repo, so it will take some weeks until I can do the update. In the meantime is there anything I can do manually to decrease the number of caps hold by the backup nodes, like flushing the client cache or something like that?

Re: [ceph-users] Is there a Ceph-mon data size partition max limit?

Den tors 9 maj 2019 kl 11:52 skrev Poncea, Ovidiu < ovidiu.pon...@windriver.com>: > Hi folks, > > What is the commanded size for the ceph-mon data partitions? Is there a > maximum limit to it? If not is there a way to limit it's growth (or celan > it up)? To my knowledge ceph-mon doesn't use a lot

Re: [ceph-users] ceph mimic and samba vfs_ceph

We maintain vfs_ceph for samba at mirror.croit.io for Debian Stretch and Buster. We apply a9c5be394da4f20bcfea7f6d4f5919d5c0f90219 on Samba 4.9 for Buster to fix this. Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 8

Re: [ceph-users] Is there a Ceph-mon data size partition max limit?

Depends on cluster size and how long you keep your cluster in a degraded state. Having ~64 GB available is a good idea Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90

Re: [ceph-users] Data moved pools but didn't move osds & backfilling+remapped loop

Hi Gregory, Sorry, I was sure I mentioned it. We installed as Luminous, upgraded to Mimic and this happend on Nautilus. (14.2.0) The data was moving until the fasthdds pool1 was "empty". The PG's do not migrate, it's going up to 377 active+clean and then the following log appears: in ceph -w: 20

Re: [ceph-users] What is recommended ceph docker image for use

Hi, Thank you. Is there a recommended version of kernel for ubuntu 16.04 to be used with ceph fs from ceph release 14 ? Sent from Mail for Windows 10 From: Stefan Kooman Sent: Thursday, May 9, 2019 7:53 AM To: Ignat Zapolsky Cc: ceph-users@lists.ceph.com Subject: Re: [ceph-users] What is recom

[ceph-users] Getting "No space left on device" when reading from cephfs

Hello I am running cephfs with 8/2 erasure coding. I had about 40tb usable free(110tb raw), one small disk crashed and i added 2x10tb disks. Now it's backfilling & recovering with 0B free and i can't read a single file from the file system... This happend with max-backfilling 4, but i have increa

Re: [ceph-users] Getting "No space left on device" when reading from cephfs

Im not sure that setup to 128 max backfills is a good idea, I shared our config for recovery and backfilling: osd recovery threads = 4 osd recovery op priority = 1 osd recovery max active = 2 osd recovery max single start = 1 osd max backfills = 4 osd backfill scan max = 16 osd backfill

Re: [ceph-users] What is recommended ceph docker image for use

It should be as recent as possible. I think would use the HWE Kernel. Ignat Zapolsky schrieb am Do., 9. Mai 2019, 13:11: > Hi, > > > > Thank you. > > > > Is there a recommended version of kernel for ubuntu 16.04 to be used with > ceph fs from ceph release 14 ? > > > > Sent from Mail

Re: [ceph-users] Getting "No space left on device" when reading from cephfs

One full OSD stops everything. You can change what's considered 'full', the default is 95% ceph osd set-full-ratio 0.95 Never let an OSD run 100% full, that will lead to lots of real problems, 95% is a good default (it's not exact, some metadata might not always be accounted or it might temporar

Re: [ceph-users] 'ceph features' showing wrong releases after upgrade to nautilus?

Use ceph versions instead Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Thu, May 9, 2019 at 9:35 AM Burkhard Linke wrote: > > Hi, > > > I've upgraded our ceph clu

Re: [ceph-users] What is recommended ceph docker image for use

Quoting Patrick Hein (bagba...@googlemail.com): > It should be as recent as possible. I think would use the HWE Kernel. ^^ This. Use linux-image-generic-hwe-16.04 (4.15 kernel). But ideally you go for Ubuntu Bionic and use linux-image-generic-hwe-18.04 (4.18 kernel). Added benefit of 4.18 kernel

Re: [ceph-users] Getting "No space left on device" when reading from cephfs

Thanks for the tips A single OSD was indeed 95% full, and after removing it there is 24TB usable space and everything working again. :D I hope during the backfilling, another OSD won't go 95% also. It's a bit odd with ~140 OSD's that a single full one can take everything down with it. I would un

[ceph-users] maximum rebuild speed for erasure coding pool

Hello all, I have a naive question about the way and the maximum rebuild speed for erasure coding pool. I did some search, but could not find any formal and detailed information about this. For pool recovering, the way Ceph works(to my understanding) is: each active OSD scrubs the drive, and if i

Re: [ceph-users] What is recommended ceph docker image for use

Thanks, that’s good. Sent from Mail for Windows 10 From: Stefan Kooman Sent: Thursday, May 9, 2019 2:42 PM To: Patrick Hein Cc: Ignat Zapolsky; ceph-users@lists.ceph.com Subject: Re: [ceph-users] What is recommended ceph docker image for use Quoting Patrick Hein (bagba...@googlemail.com): > It s

Re: [ceph-users] maximum rebuild speed for erasure coding pool

Den tors 9 maj 2019 kl 15:46 skrev Feng Zhang : > > For erasure pool, suppose I have 10 nodes, each has 10 6TB drives, so > in total 100 drives. I make a 4+2 erasure pool, failure domain is > host/node. Then if one drive failed, (assume the 6TB is fully used), > what the maximum speed the recoveri

Re: [ceph-users] 'ceph features' showing wrong releases after upgrade to nautilus?

Hi Paul, On 5/9/19 3:27 PM, Paul Emmerich wrote: Use ceph versions instead Thanks, ceph versions gives the right output. Regards, Burkhard ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.

Re: [ceph-users] maximum rebuild speed for erasure coding pool

> Fancy fast WAL/DB/Journals probably help a lot here, since they do affect the "iops" > you experience from your spin-drive OSDs. What difference can be expected if you have a 100 iops hdd and you start using wal/db/journals on ssd? What would this 100 iops increase to (estimating)? --

Re: [ceph-users] maximum rebuild speed for erasure coding pool

Den tors 9 maj 2019 kl 16:17 skrev Marc Roos : > > > Fancy fast WAL/DB/Journals probably help a lot here, since they do > affect the "iops" > > you experience from your spin-drive OSDs. > > What difference can be expected if you have a 100 iops hdd and you start > using > wal/db/journals on ssd?

Re: [ceph-users] maximum rebuild speed for erasure coding pool

Thanks, guys. I forgot the IOPS. So since I have 100disks, the total IOPS=100X100=10K. For the 4+2 erasure, one disk fail, then it needs to read 5 and write 1 objects.Then the whole 100 disks can do 10K/6 ~ 2K rebuilding actions per seconds. While for the 100X6TB disks, suppose the object size is

[ceph-users] PG in UP set but not Acting? Backfill halted

Hello, Been working on Ceph for only a few weeks and have a small cluster in VMs. I did a ceph-ansible rolling_update to nautilus from mimic and some of my PG were stuck in 'active+undersized+remapped+backfilling' with no progress. All OSD were up and in (see ceph tree below). The PGs only had 2

Re: [ceph-users] Ceph Bucket strange issues rgw.none + id and marker diferent.

Hi Manuel, I’ve interleaved responses below. > On May 8, 2019, at 3:17 PM, EDH - Manuel Rios Fernandez > wrote: > > Eric, > > Yes we do : > > time s3cmd ls s3://[BUCKET]/ --no-ssl and we get near 2min 30 secs for list > the bucket. > > If we instantly hit again the query it normally timeou

Re: [ceph-users] What is recommended ceph docker image for use

Thank you. With regards, Ignat Zapolsky Sent from my iPhone > On 9 May 2019, at 14:42, Stefan Kooman wrote: > > Quoting Patrick Hein (bagba...@googlemail.com): >> It should be as recent as possible. I think would use the HWE Kernel. > > ^^ This. Use linux-image-generic-hwe-16.04 (4.15 kernel).

Re: [ceph-users] Clients failing to respond to cache pressure

On Thu, May 9, 2019 at 3:21 AM Stolte, Felix wrote: > > Thanks for the info Patrick. We are using ceph packages from ubuntu main > repo, so it will take some weeks until I can do the update. In the meantime > is there anything I can do manually to decrease the number of caps hold by > the backu

Re: [ceph-users] ceph-create-keys loops

Update: deployment (ansible 2.6 + ceph-ansible 3.2) completed after cleaning up everything deployed before. Thanks a lot. From: ceph-users On Behalf Of ST Wong (ITSC) Sent: Tuesday, May 7, 2019 6:22 PM To: solarflow99 ; ceph-users@lists.ceph.com Subject: Re: [ceph-users] ceph-create-keys loops

Re: [ceph-users] RGW Bucket unable to list buckets 100TB bucket

Hi Yesterday night we added 2 Intel Optane Nvme Generated 4 partitions for get the max performance (Q=32) of those monsters, total 8 Partitions of 50GB. Move the rgw.index pool got filled near 3GB . And... Still the same issue, listing buckets its really slow or deeply slow that make its u