Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-19 Thread Nikola Ciprich
> > opts="--randrepeat=1 --ioengine=rbd --direct=1 --numjobs=${numjobs} > > --gtod_reduce=1 --name=test --pool=${pool} --rbdname=${vol} --invalidate=0 > > --bs=4k --iodepth=64 --time_based --runtime=$time --group_reporting" > > > > So that "--numjobs" parameter is what I was referring to when I sa

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Alexandre DERUMIER
t cpu frequency, disable rbd cache, disable client debug, or others option which can lower cpu usage client side. - Mail original - De: "Nikola Ciprich" À: "ceph-users" Cc: "nik" Envoyé: Mercredi 18 Juillet 2018 16:54:58 Objet: [ceph-users] krbd vs librbd p

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Jason Dillaman
On Wed, Jul 18, 2018 at 1:08 PM Nikola Ciprich wrote: > > Care to share your "bench-rbd" script (on pastebin or similar)? > sure, no problem.. it's so short I hope nobody will get offended if I > paste it right > here :) > > #!/bin/bash > > #export LD_PRELOAD="/usr/lib64/libtcmalloc.so.4" > numjo

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Nikola Ciprich
> Care to share your "bench-rbd" script (on pastebin or similar)? sure, no problem.. it's so short I hope nobody will get offended if I paste it right here :) #!/bin/bash #export LD_PRELOAD="/usr/lib64/libtcmalloc.so.4" numjobs=8 pool=nvme vol=xxx time=30 opts="--randrepeat=1 --ioengine=rbd --d

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Jason Dillaman
On Wed, Jul 18, 2018 at 12:58 PM Nikola Ciprich wrote: > > What's the output from "rbd info nvme/centos7"? > that was it! the parent had some of unsupported features > enabled, therefore the child could not be mapped.. > > so the error message is a bit confusing, but now after disabling > the fea

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Nikola Ciprich
> What's the output from "rbd info nvme/centos7"? that was it! the parent had some of unsupported features enabled, therefore the child could not be mapped.. so the error message is a bit confusing, but now after disabling the features on the parent it works for me, thanks! > Odd. The exclusive-l

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Jason Dillaman
On Wed, Jul 18, 2018 at 12:36 PM Nikola Ciprich wrote: > ;6QHi Janon, > > > Just to clarify: modern / rebased krbd block drivers definitely support > > layering. The only missing features right now are object-map/fast-diff, > > deep-flatten, and journaling (for RBD mirroring). > > I thought it as

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Nikola Ciprich
;6QHi Janon, > Just to clarify: modern / rebased krbd block drivers definitely support > layering. The only missing features right now are object-map/fast-diff, > deep-flatten, and journaling (for RBD mirroring). I thought it as well, but at least mapping clone does not work for me even under 4.1

Re: [ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Jason Dillaman
On Wed, Jul 18, 2018 at 10:55 AM Nikola Ciprich wrote: > Hi, > > historically I've found many discussions about this topic in > last few years, but it seems to me to be still a bit unresolved > so I'd like to open the question again.. > > In all flash deployments, under 12.2.5 luminous and qemu

[ceph-users] krbd vs librbd performance with qemu

2018-07-18 Thread Nikola Ciprich
Hi, historically I've found many discussions about this topic in last few years, but it seems to me to be still a bit unresolved so I'd like to open the question again.. In all flash deployments, under 12.2.5 luminous and qemu 12.2.0 using lbirbd, I'm getting much worse results regarding IOPS th