[ceph-users] Re: Small HDD cluster, switch from Bluestore to Filestore

2019-08-14 Thread Rich Bade
Thanks Robert, I'm trying those settings to see if they make a difference for our case. It's usually around the weekend we have issues so should have some idea by next week. ___ ceph-users mailing list -- ceph-users@ceph.io To unsubscribe send an email

[ceph-users] "Signature check failed" from certain clients

2019-08-14 Thread Peter Sarossy
hey folks, I spent the past 2 hours digging through the forums and similar sources with no luck.. I use ceph storage for docker stacks, and this issue has taken the whole thing down as I cannot mount their volumes back... Starting yesterday, some of my nodes cannot mount the filesystem and it j

[ceph-users] Re: Mgr stability

2019-08-14 Thread Reed Dier
Thanks for that insight. My main metrics source is the influx plugin, but I enabled the prometheus plugin to get access to the per-rbd image metrics. I may disable prometheus and see if that yields better stability, until possibly the influx plugin gets updated to support those metric exports.

[ceph-users] Re: Mgr stability

2019-08-14 Thread shubjero
I'm having a similar issue with ceph-mgr stability problems since upgrading from 13.2.5 to 13.2.6. I have isolated the crashing to the prometheus module being enabled and notice much better stability when the prometheus module is NOT enabled. No more failovers, however I do notice that even with pr

[ceph-users] Re: Failing heartbeats when no backfill is running

2019-08-14 Thread Lorenz Kiefner
Ok, then Ceph probably doesn't fit for me. I wanted to provide a backup platform for me, my family and my friends. Speed is not relevant, but long-term reliability. So I'm depending on home internet connections and VPN. At the moment I'm mostly using wireguard, but I could switch to openvpn to re

[ceph-users] Mgr stability

2019-08-14 Thread Reed Dier
Hi all, I'm experiencing some stability issues with my ceph-mgr daemons post Nautilus upgrade (13.2.6->14.2.2). On top of Ubuntu 18.04, the mgr's are colocated with the mon's, and 2 of the 3 mon/mgr nodes also colocate an MDS, running a single rank active:standby-replay. The main issue(s) I am

[ceph-users] Re: Fw: Ceph-Deploy

2019-08-14 Thread Paul Emmerich
You cannot add OSDs to a cluster that is offline. -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Wed, Aug 14, 2019 at 5:43 PM Cory Mueller wrote: > > Good Morning, > > >

[ceph-users] Re: rbd image usage per osd

2019-08-14 Thread Mykola Golub
On Fri, Aug 09, 2019 at 12:44:42PM -0400, Frank R wrote: > I have an all RBD pool/cluster. I am interested in tracking how much disk > space is being used by each RBD image on every OSD drive. > > The OSDs are Filestore. > > Does anyone know of any existing scripts that accomplish this task? > >

[ceph-users] Re: Failing heartbeats when no backfill is running

2019-08-14 Thread Wido den Hollander
On 8/14/19 5:46 PM, Lorenz Kiefner wrote: > Hi, > > this was the first thing I was thinking about (and yes, there had been > some issues, but they are resolved - double checked!). > > MTU is consistent throughout the whole net and pings in all sizes are > handled well. And MTU problems wouldn'

[ceph-users] Re: Failing heartbeats when no backfill is running

2019-08-14 Thread Lorenz Kiefner
Hi, this was the first thing I was thinking about (and yes, there had been some issues, but they are resolved - double checked!). MTU is consistent throughout the whole net and pings in all sizes are handled well. And MTU problems wouldn't probably make a difference between backfills and normal o

[ceph-users] Fw: Ceph-Deploy

2019-08-14 Thread Cory Mueller
Good Morning, I was wondering if you were aware of a way to add more OSD nodes to an existing OFFLINE cluster. Ive tried to use ceph-deploy new ceph-deploy admin ceph-deploy install and none of them allow me to join the cluster. I

[ceph-users] Re: Mapped rbd is very slow

2019-08-14 Thread Olivier AUDRY
hello I mean a filesystem mounted on top of a mapped rbd rbd create --size=10G kube/benchrbd feature disable kube/bench object- map fast-diff deep-flattenrbd map bench --pool kube --name client.admin/sbin/mkfs.ext4 /dev/rbd/kube/bench mount /dev/rbd/kube/bench /mnt/cd /mnt/ about the bench I did

[ceph-users] Re: Mapped rbd is very slow

2019-08-14 Thread Ilya Dryomov
On Wed, Aug 14, 2019 at 2:49 PM Paul Emmerich wrote: > > On Wed, Aug 14, 2019 at 2:38 PM Olivier AUDRY wrote: > > let's test random write > > rbd -p kube bench kube/bench --io-type write --io-size 8192 --io-threads > > 256 --io-total 10G --io-pattern rand > > elapsed: 125 ops: 1310720 ops/s

[ceph-users] Re: Mapped rbd is very slow

2019-08-14 Thread Paul Emmerich
On Wed, Aug 14, 2019 at 2:38 PM Olivier AUDRY wrote: > let's test random write > rbd -p kube bench kube/bench --io-type write --io-size 8192 --io-threads 256 > --io-total 10G --io-pattern rand > elapsed: 125 ops: 1310720 ops/sec: 10416.31 bytes/sec: 85330446.58 > > dd if=/dev/zero of=test b

[ceph-users] Mapped rbd is very slow

2019-08-14 Thread Olivier AUDRY
hello I got a huge difference of read/write performance on my ceph cluster when I use rbd. rbd bench reach the limit of my cluster (1gbps network) when the performance in mapped rbd is very low. 30MB/s. ceph version : ceph version 14.2.2 (4f8fa0a0024755aae7d95567c63f11d6862d55be) nautilus (stabl

[ceph-users] Re: Failing heartbeats when no backfill is running

2019-08-14 Thread Paul Emmerich
MTU issues due to the VPN connection? Paul -- Paul Emmerich Looking for help with your Ceph cluster? Contact us at https://croit.io croit GmbH Freseniusstr. 31h 81247 München www.croit.io Tel: +49 89 1896585 90 On Wed, Aug 14, 2019 at 1:48 PM Lorenz Kiefner wrote: > > Dear ceph-users, > > I

[ceph-users] Failing heartbeats when no backfill is running

2019-08-14 Thread Lorenz Kiefner
Dear ceph-users, I'm having trouble with heartbeats, there are a lot of "heartbeat_check: no reply from..."-messages in my logs when there is no backfilling or repairing running (yes, it's failing when all PGs are active+clean). Only a few OSDs are failing, even when there are several OSDs on the

[ceph-users] Re: BlueStore _txc_add_transaction errors (possibly related to bug #38724)

2019-08-14 Thread Florian Haas
On 12/08/2019 21:07, Alexandre Marangone wrote: >> rados -p volumes stat 'obj-vS6RN9\uQwvXU9DP' >> error stat-ing volumes/obj-vS6RN9\uQwvXU9DP: (2) No such file or directory > I believe you need to substitute \u with _ Yes indeed, thank you! Cheers, Florian __

[ceph-users] Re: BlueStore _txc_add_transaction errors (possibly related to bug #38724)

2019-08-14 Thread Florian Haas
Hi Tom, responding back on this briefly so that people are in the loop; I'll have more details in a blog post that I hope to get around to writing. On 12/08/2019 11:34, Thomas Byrne - UKRI STFC wrote: >> And bluestore should refuse to start if the configured limit is > 4GB. Or >> something alon