Re: [ceph-users] slow requests and high i/o / read rate on bluestore osds after upgrade 12.2.8 -> 12.2.10

2019-01-19 Thread Mykola Golub
On Fri, Jan 18, 2019 at 11:06:54AM -0600, Mark Nelson wrote: > IE even though you guys set bluestore_cache_size to 1GB, it is being > overridden by bluestore_cache_size_ssd. Isn't it vice versa [1]? [1] https://github.com/ceph/ceph/blob/luminous/src/os/bluestore/BlueStore.cc#L3976 -- Mykola G

Re: [ceph-users] Ceph MDS laggy

2019-01-19 Thread Adam Tygart
It worked for about a week, and then seems to have locked up again. Here is the back trace from the threads on the mds: http://people.cs.ksu.edu/~mozes/ceph-12.2.10-laggy-mds.gdb.txt -- Adam On Sun, Jan 13, 2019 at 7:41 PM Yan, Zheng wrote: > > On Sun, Jan 13, 2019 at 1:43 PM Adam Tygart wrote

Re: [ceph-users] Boot volume on OSD device

2019-01-19 Thread Brian Topping
> On Jan 18, 2019, at 10:58 AM, Hector Martin wrote: > > Just to add a related experience: you still need 1.0 metadata (that's > the 1.x variant at the end of the partition, like 0.9.0) for an > mdadm-backed EFI system partition if you boot using UEFI. This generally > works well, except on some

Re: [ceph-users] Ceph MDS laggy

2019-01-19 Thread Paul Emmerich
Did this only start to happen after upgrading to 12.2.10? Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Sat, Jan 19, 2019 at 5:40 PM Adam Tygart wrote: > > It wor

Re: [ceph-users] Ceph MDS laggy

2019-01-19 Thread Adam Tygart
Yes, we upgraded to 12.2.10 from 12.2.7 on the 27th of December. This didn't happen before then. -- Adam On Sat, Jan 19, 2019, 20:17 Paul Emmerich mailto:paul.emmer...@croit.io> wrote: Did this only start to happen after upgrading to 12.2.10? Paul -- Paul Emmerich Looking for help with your

Re: [ceph-users] Ceph MDS laggy

2019-01-19 Thread Adam Tygart
Just re-checked my notes. We updated from 12.2.8 to 12.2.10 on the 27th of December. -- Adam On Sat, Jan 19, 2019 at 8:26 PM Adam Tygart wrote: > > Yes, we upgraded to 12.2.10 from 12.2.7 on the 27th of December. This didn't > happen before then. > > -- > Adam > > On Sat, Jan 19, 2019, 20:17 Pa

Re: [ceph-users] Ceph Nautilus Release T-shirt Design

2019-01-19 Thread Alex Gorbachev
On Wed, Jan 16, 2019 at 12:46 PM Mike Perez wrote: > > Hey everyone, > > We're getting close to the release of Ceph Nautilus, and I wanted to > start the discussion of our next shirt! > > It looks like in the past we've used common works from Wikipedia pages. > > https://en.wikipedia.org/wiki/Naut

Re: [ceph-users] Ceph MDS laggy

2019-01-19 Thread Adam Tygart
The same user's jobs seem to be the instigator of this issue again. I've looked through their code and see nothing too onerous. This time it was 2400+ cores/jobs on 186 nodes all working in the same directory. Each job reads in a different 110KB file, crunches numbers for while (1+ hours) and then

[ceph-users] Salvage CEPHFS after lost PG

2019-01-19 Thread Rik
Hi all, I'm looking for some suggestions on how to do something inappropriate. In a nutshell, I've lost the WAL/DB for three bluestore OSDs on a small cluster and, as a result of those three OSDs going offline, I've lost a placement group (7.a7). How I achieved this feat is an embarrassing mis