Re: [ceph-users] rados block on SSD - performance - how to tune and get insight?

2019-02-06 Thread Christian Balzer
o the number Micron are reporting on NVMe? > > Thanks a log. > > [0] > https://www.micron.com/-/media/client/global/documents/products/other-documents/micron_9200_max_ceph_12,-d-,2,-d-,8_luminous_bluestore_reference_architecture.pdf?la=en > > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Rakuten Communications ___ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

Re: [ceph-users] Prevent rebalancing in the same host?

2019-02-19 Thread Christian Balzer
ance ever you could also permanently set noout and nodown and live with the consequences and warning state. But of course everybody will (rightly) tell you that you need enough capacity to at the very least deal with a single OSD loss. Christian -- Christian BalzerNetwork/Systems Engineer

Re: [ceph-users] Ceph cluster on AMD based system.

2019-03-05 Thread Christian Balzer
; >> On 3/5/19 3:49 AM, Darius Kasparavičius wrote: > >>> Hello, > >>> > >>> > >>> I was thinking of using AMD based system for my new nvme based > >>> cluster. In particular I'm looking at > >>> https://www.supermi

[ceph-users] Dealing with SATA resets and consequently slow ops

2019-03-26 Thread Christian Balzer
133 SControl 300) [54954737.206133] ata5.00: configured for UDMA/133 [54954737.206140] ata5: EH complete --- -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Rakuten Communications ___ ceph-users mailing

Re: [ceph-users] Bluestore WAL/DB decisions

2019-03-28 Thread Christian Balzer
considered. > > Is the penalty for a too small DB on an SSD partition so severe that > it's not worth doing? > > Thanks, > Erik > _______ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lis

[ceph-users] Erasure Coding failure domain (again)

2019-03-31 Thread Christian Balzer
uce things down to the same risk as a 3x replica pool. Feedback welcome. Christian -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Rakuten Communications ___ ceph-users mailing list ceph-users@lists.cep

Re: [ceph-users] Ceph block storage cluster limitations

2019-03-31 Thread Christian Balzer
es. > I'm happy that somebody else spotted this. ^o^ Regards, Christian > > What is the known maximum cluster size that Ceph RBD has been deployed to? > > See above. > ___ > ceph-users mailing list > ceph-users

Re: [ceph-users] Does Bluestore backed OSD detect bit rot immediately when reading or only when scrubbed?

2019-03-31 Thread Christian Balzer
red" semi-automatically with ceph pg repair? > > What and how would happen in case erasure coded pool's data was found > to be damaged as well? > > -- > End of message. Next message? > ___ > ceph-users mailing list > ceph-users@lists.ceph.com > http://lists.ceph.

Re: [ceph-users] Erasure Coding failure domain (again)

2019-04-02 Thread Christian Balzer
Hello Hector, Firstly I'm so happy somebody actually replied. On Tue, 2 Apr 2019 16:43:10 +0900 Hector Martin wrote: > On 31/03/2019 17.56, Christian Balzer wrote: > > Am I correct that unlike with with replication there isn't a maximum size > > of the critical path

Re: [ceph-users] Erasure Coding failure domain (again)

2019-04-02 Thread Christian Balzer
On Tue, 2 Apr 2019 19:04:28 +0900 Hector Martin wrote: > On 02/04/2019 18.27, Christian Balzer wrote: > > I did a quick peek at my test cluster (20 OSDs, 5 hosts) and a replica 2 > > pool with 1024 PGs. > > (20 choose 2) is 190, so you're never going to have more tha

Re: [ceph-users] RGW: Reshard index of non-master zones in multi-site

2019-04-07 Thread Christian Balzer
> I would assume then that unlike what documentation says, it's safe to > > run 'reshard stale-instances rm' on a multi-site setup. > > > > However it is quite telling if the author of this feature doesn't > > trust what they have written to work co

Re: [ceph-users] Erasure Coding failure domain (again)

2019-04-10 Thread Christian Balzer
week" situation like experienced with several people here, you're even more like to wind up in trouble very fast. This is of course all something people do (or should know), I'm more wondering how to model it to correctly asses risks. Christian On Wed, 3 Apr 2019 10:28:09 +0900 Ch

Re: [ceph-users] Erasure Coding failure domain (again)

2019-04-10 Thread Christian Balzer
Hello, On Wed, 10 Apr 2019 20:09:58 +0200 Paul Emmerich wrote: > On Wed, Apr 10, 2019 at 11:12 AM Christian Balzer wrote: > > > > > > Hello, > > > > Another thing that crossed my mind aside from failure probabilities caused > > by actual HDDs dying i

Re: [ceph-users] How to reduce HDD OSD flapping due to rocksdb compacting event?

2019-04-11 Thread Christian Balzer
ou're using object store), how busy those disks and CPUs are, etc. That kind of information will be invaluable for others here and likely the developers as well. Regards, Christian > Kind regards, > > Charles Alva > Sent from Gmail Mobile -- Christian BalzerNe

Re: [ceph-users] how to judge the results? - rados bench comparison

2019-04-17 Thread Christian Balzer
But only completely so if everything is on the same boat. So if you clients (or most of them at least) can be on 25GB/s as well, that would be the best situation, with a non-split network. Christian > > > > > My 2 cents, > > > > Gr. Stefan > > > > Cheers, &

Re: [ceph-users] how to judge the results? - rados bench comparison

2019-04-17 Thread Christian Balzer
are you probably want to reduce > > recovery speed anyways if you would run into that limit > > > > Paul > > > > Lars > _______ > ceph-users mailing list > ceph-users@lists.ceph.com > http://

Re: [ceph-users] how to judge the results? - rados bench comparison

2019-04-17 Thread Christian Balzer
On Wed, 17 Apr 2019 16:08:34 +0200 Lars Täuber wrote: > Wed, 17 Apr 2019 20:01:28 +0900 > Christian Balzer ==> Ceph Users : > > On Wed, 17 Apr 2019 11:22:08 +0200 Lars Täuber wrote: > > > > > Wed, 17 Apr 2019 10:47:32 +0200 > > > Paul Emmerich ==

Re: [ceph-users] Unexpected IOPS Ceph Benchmark Result

2019-04-21 Thread Christian Balzer
7;m a little bit confuse now. I suppose to get different results when using > different pool image, but it isnt. It's like using 1 same performance. > Although we're really sure that we alreay separate the SSD and HDD pool and > crushmap. > > My question is : > > 1. W

Re: [ceph-users] Package availability for Debian / Ubuntu

2019-05-16 Thread Christian Balzer
jessie has 1 Packages. No ceph package found > stretch has 1 Packages. No ceph package found > > If you want to re-run these tests, the attached hacky shell script does it. > > Regards, > > Matthew > > > > -- > The Wellcome Sanger Institute is operated by Genome R

Re: [ceph-users] HEALTH_ERR with a kitchen sink of problems: MDS damaged, readonly, and so forth

2019-07-24 Thread Christian Balzer
On Thu, 25 Jul 2019 13:49:22 +0900 Sangwhan Moon wrote: > osd: 39 osds: 39 up, 38 in You might want to find that out OSD. -- Christian BalzerNetwork/Systems Engineer ch...@gol.com Rakuten Mobile Inc. ___ ceph-us

[ceph-users] Bluestore caching oddities, again

2019-08-03 Thread Christian Balzer
Reads from a hot cache with direct=0 read: IOPS=199, BW=797MiB/s (835MB/s)(32.0GiB/41130msec) with direct=1 read: IOPS=702, BW=2810MiB/s (2946MB/s)(32.0GiB/11662msec) Which is as fast as gets with this setup. Comments? Christian -- Christian BalzerNetwork/Systems Engineer

Re: [ceph-users] Bluestore caching oddities, again

2019-08-04 Thread Christian Balzer
Hello, On Sun, 4 Aug 2019 06:34:46 -0500 Mark Nelson wrote: > On 8/4/19 6:09 AM, Paul Emmerich wrote: > > > On Sun, Aug 4, 2019 at 3:47 AM Christian Balzer wrote: > > > >> 2. Bluestore caching still broken > >> When writing data with the fios below, it

Re: [ceph-users] SPAM in the ceph-users list

2019-11-12 Thread Christian Balzer
labor intensive and a nuisance for real users) as well as harsher ingress and egress (aka spamfiltering) controls you will find that all the domains spamvertized are now in the Spamhaus DBL. "host abbssm.edu.in.dbl.spamhaus.org" Pro tip for spammers: Don't get my attention, ever. Ch

Re: [ceph-users] Slow rbd read performance

2019-12-23 Thread Christian Balzer
c: 5517.19 bytes/sec: > 45196784.26 (45MB/sec) => WHY JUST 45MB/sec? > > Since i ran those rbd benchmarks in ceph01, i guess the problem is not > related to my backup rbd mount at all? > > Thanks, > Mario > ___ > ceph

Re: [ceph-users] Slow rbd read performance

2019-12-26 Thread Christian Balzer
2 sec at 164MiB/sec 41 > IOPS > osd.27: bench: wrote 1GiB in blocks of 4MiB in 7.00978 sec at 146MiB/sec 36 > IOPS > osd.32: bench: wrote 1GiB in blocks of 4MiB in 6.38438 sec at 160MiB/sec 40 > IOPS > > Thanks, > Mario > > > > On Tue, Dec 24, 2019 at 1:46 A

Re: [ceph-users] Slow Performance - Sequential IO

2020-01-17 Thread Christian Balzer
> random write. No high CPU load/interface saturation is noted when running > tests against the rbd. > > > > When testing with a 4K block size against an RBD on a dedicated metal test > host (same specs as other cluster nodes noted above) I get the following > (c

<    8   9   10   11   12   13