Re: [ceph-users] I/O Speed Comparisons

2013-04-22 Thread Wolfgang Hennerbichler
On 04/16/2013 07:41 PM, Josh Durgin wrote: > Hmm, good to know. Do you know what the qemu issue was? Sorry for the late reply. It had to do with the setting. When I had pc-1.0 in there, it behaved as if the whole CPU was emulateed in software. I could look at the kernel log messages while boot

Re: [ceph-users] I/O Speed Comparisons

2013-04-16 Thread Josh Durgin
On 04/16/2013 02:08 AM, Wolfgang Hennerbichler wrote: On 03/29/2013 09:46 PM, Josh Durgin wrote: The issue was that the qemu rbd driver was blocking the main qemu thread when flush was called, since it was using a synchronous flush. Fixing this involves patches to librbd to add an asynchronous

Re: [ceph-users] I/O Speed Comparisons

2013-04-16 Thread Wolfgang Hennerbichler
On 03/29/2013 09:46 PM, Josh Durgin wrote: > The issue was that the qemu rbd driver was blocking the main qemu > thread when flush was called, since it was using a synchronous flush. > Fixing this involves patches to librbd to add an asynchronous flush, > and a patch to qemu to use it. > > The ce

Re: [ceph-users] I/O Speed Comparisons

2013-04-01 Thread Wolfgang Hennerbichler
On Fri, Mar 29, 2013 at 01:46:16PM -0700, Josh Durgin wrote: > The issue was that the qemu rbd driver was blocking the main qemu > thread when flush was called, since it was using a synchronous flush. > Fixing this involves patches to librbd to add an asynchronous flush, > and a patch to qemu to us

Re: [ceph-users] I/O Speed Comparisons

2013-03-29 Thread Josh Durgin
On 03/19/2013 07:06 PM, Josh Durgin wrote: On 03/18/2013 07:53 AM, Wolfgang Hennerbichler wrote: On 03/13/2013 06:38 PM, Josh Durgin wrote: Anyone seeing this problem, could you try the wip-rbd-cache-aio branch? Hi, just compiled and tested it out, unfortunately there's no big change

Re: [ceph-users] I/O Speed Comparisons

2013-03-19 Thread Josh Durgin
On 03/18/2013 07:53 AM, Wolfgang Hennerbichler wrote: On 03/13/2013 06:38 PM, Josh Durgin wrote: Anyone seeing this problem, could you try the wip-rbd-cache-aio branch? Hi, just compiled and tested it out, unfortunately there's no big change: ceph --version ceph version 0.58-375-ga4a6075

Re: [ceph-users] I/O Speed Comparisons

2013-03-18 Thread Wolfgang Hennerbichler
On 03/13/2013 06:38 PM, Josh Durgin wrote: > Anyone seeing this problem, could you try the wip-rbd-cache-aio branch? Hi, just compiled and tested it out, unfortunately there's no big change: ceph --version ceph version 0.58-375-ga4a6075 (a4a60758b7a10d51419c1961bb12f26b87672fd5) libvirt-conf

Re: [ceph-users] I/O Speed Comparisons

2013-03-14 Thread Wolfgang Hennerbichler
-- Sent from my mobile device On 13.03.2013, at 18:38, "Josh Durgin" wrote: > On 03/12/2013 12:46 AM, Wolfgang Hennerbichler wrote: >> >> >> On 03/11/2013 11:56 PM, Josh Durgin wrote: >> dd if=/dev/zero of=/bigfile bs=2M & Serial console gets jerky, VM gets unresponsive. It

Re: [ceph-users] I/O Speed Comparisons

2013-03-13 Thread Josh Durgin
On 03/12/2013 12:46 AM, Wolfgang Hennerbichler wrote: On 03/11/2013 11:56 PM, Josh Durgin wrote: dd if=/dev/zero of=/bigfile bs=2M & Serial console gets jerky, VM gets unresponsive. It doesn't crash, but it's not 'healthy' either. CPU load isn't very high, it's in the waiting state a lot:

Re: [ceph-users] I/O Speed Comparisons

2013-03-12 Thread Christopher Kunz
Am 12.03.13 06:08, schrieb Stefan Priebe - Profihost AG: > Hi Sage, > > i would like to see the high ping problem fixed in 0.56.4 > > Thanks! > For the record, we (filoo) reported this issue about 3 months ago. --ck ___ ceph-users mailing list ceph-u

Re: [ceph-users] I/O Speed Comparisons

2013-03-12 Thread Wolfgang Hennerbichler
On 03/11/2013 11:56 PM, Josh Durgin wrote: >> dd if=/dev/zero of=/bigfile bs=2M & >> >> Serial console gets jerky, VM gets unresponsive. It doesn't crash, but >> it's not 'healthy' either. CPU load isn't very high, it's in the waiting >> state a lot: > > Does this only happen with rbd_cache tur

Re: [ceph-users] I/O Speed Comparisons

2013-03-11 Thread Andrew Thrift
det: Samstag, 09. März 2013 20:33 Bis: ceph-users@lists.ceph.com Betreff: Re: [ceph-users] I/O Speed Comparisons Thanks for all of this feedback guys! It gives us some good data to try to replicate on our end. Hopefully I'll have some time next week to take a look. Thanks! Mark On 03/09/2013 0

Re: [ceph-users] I/O Speed Comparisons

2013-03-11 Thread Stefan Priebe - Profihost AG
n 03/11/2013 01:30 AM, Wolfgang Hennerbichler wrote: >>>> Let me know if I can help out with testing somehow. >>>> >>>> Wolfgang >>>> >>>> Von: ceph-users-boun...@lists.ceph.com >>>> [c

Re: [ceph-users] I/O Speed Comparisons

2013-03-11 Thread Josh Durgin
.ceph.com]" im Auftrag von "Mark Nelson [mark.nel...@inktank.com] Gesendet: Samstag, 09. März 2013 20:33 Bis: ceph-users@lists.ceph.com Betreff: Re: [ceph-users] I/O Speed Comparisons Thanks for all of this feedback guys! It gives us some good data to try to replicate on our end. Hopef

Re: [ceph-users] I/O Speed Comparisons

2013-03-11 Thread Wolfgang Hennerbichler
me know if I can help out with testing somehow. >> >> Wolfgang >> >> Von: ceph-users-boun...@lists.ceph.com >> [ceph-users-boun...@lists.ceph.com]" im Auftrag von "Mark >> Nelson [mark.nel...@inktank.com] >> Gesendet: Samstag, 09

Re: [ceph-users] I/O Speed Comparisons

2013-03-11 Thread Mark Nelson
Von: ceph-users-boun...@lists.ceph.com [ceph-users-boun...@lists.ceph.com]" im Auftrag von "Mark Nelson [mark.nel...@inktank.com] Gesendet: Samstag, 09. März 2013 20:33 Bis: ceph-users@lists.ceph.com Betreff: Re: [ceph-users] I/O Speed Comparisons Thanks for all of this feedback

Re: [ceph-users] I/O Speed Comparisons

2013-03-10 Thread Wolfgang Hennerbichler
ceph-users@lists.ceph.com Betreff: Re: [ceph-users] I/O Speed Comparisons Thanks for all of this feedback guys! It gives us some good data to try to replicate on our end. Hopefully I'll have some time next week to take a look. Thanks! Mark On 03/09/2013 08:14 AM, Erdem Agaoglu wrote: > Mark, &g

Re: [ceph-users] I/O Speed Comparisons

2013-03-09 Thread Mark Nelson
Thanks for all of this feedback guys! It gives us some good data to try to replicate on our end. Hopefully I'll have some time next week to take a look. Thanks! Mark On 03/09/2013 08:14 AM, Erdem Agaoglu wrote: Mark, If it's any help, we've done a small totally unreliable benchmark on our

Re: [ceph-users] I/O Speed Comparisons

2013-03-09 Thread Erdem Agaoglu
Mark, If it's any help, we've done a small totally unreliable benchmark on our end. For a KVM instance, we had: 260MB/s write, 200MB/s read on local SAS disks, attached as LVM LVs, 250MB/s write, 90MB/s read on RBD, 32 osds, all SATA. All sequential, a 10G network. It's more than enough currently

Re: [ceph-users] I/O Speed Comparisons

2013-03-08 Thread Andrew Thrift
Mark, I would just like to add, we too are seeing the same behavior with QEMU/KVM/RBD. Maybe it is a common symptom of high IO with this setup. Regards, Andrew On 3/8/2013 12:46 AM, Mark Nelson wrote: On 03/07/2013 05:10 AM, Wolfgang Hennerbichler wrote: On 03/06/2013 02:31 PM, M

Re: [ceph-users] I/O Speed Comparisons

2013-03-07 Thread Wolfgang Hennerbichler
On 03/07/2013 12:46 PM, Mark Nelson wrote: > Thanks for the heads up Wolfgang. I'm going to be looking into QEMU/KVM > RBD performance in the coming weeks so I'll try to watch out for this > behaviour. Thanks for taking the time. It seems to me as if there are so many interrupts within the vir

Re: [ceph-users] I/O Speed Comparisons

2013-03-07 Thread Mark Nelson
On 03/07/2013 05:10 AM, Wolfgang Hennerbichler wrote: On 03/06/2013 02:31 PM, Mark Nelson wrote: t If you are doing sequential reads, you may benefit by increasing the read_ahead_kb value for each device in /sys/block//queue on the OSD hosts. Thanks, that didn't really help. It seems the VM

Re: [ceph-users] I/O Speed Comparisons

2013-03-07 Thread Wolfgang Hennerbichler
On 03/06/2013 02:31 PM, Mark Nelson wrote: > If you are doing sequential reads, you may benefit by increasing the > read_ahead_kb value for each device in /sys/block//queue on the > OSD hosts. Thanks, that didn't really help. It seems the VM has to handle too much I/O, even the mouse-cursor is

Re: [ceph-users] I/O Speed Comparisons

2013-03-06 Thread Mark Nelson
On 03/06/2013 07:28 AM, Wolfgang Hennerbichler wrote: Hi, I've set up (latest bobtail) ceph between 2 Machines, 2 OSDs each. Then I've started up a VM backed by LVM with windows something in it, and i ran the crappy ATTO Disk Benchmark to get performance information (my customer wants me to do i