[ceph-users] OSD Performance Counters

2016-02-18 Thread Nick Fisk
Hi All, Could someone please sanity check this for me please. I trying to get my head round what counter reflect what and how they correlate to end user performance. In the attached graph I am graphing averages of the counters across all OSD's on one host Blue = osd.w_op_latency Red = Max of abo

Re: [ceph-users] OSD Performance

2015-02-24 Thread Christian Balzer
Hello Kevin, On Wed, 25 Feb 2015 07:55:34 +0400 Kevin Walker wrote: > Hi Christian > > We are just looking at options at this stage. > Never a bad thing to do. > Using a hardware RAM disk for the journal is the same concept as the > SolidFire guys, who are also using XFS (at least they were

Re: [ceph-users] OSD Performance

2015-02-24 Thread Kevin Walker
Hi Christian We are just looking at options at this stage. Using a hardware RAM disk for the journal is the same concept as the SolidFire guys, who are also using XFS (at least they were last time I crossed paths with a customer using SolidFire) and from my experiences with ZFS, using a RAM ba

Re: [ceph-users] OSD Performance

2015-02-24 Thread Christian Balzer
On Wed, 25 Feb 2015 02:50:59 +0400 Kevin Walker wrote: > Hi Mark > > Thanks for the info, 22k is not bad, but still massively below what a > pcie ssd can achieve. Care to expand on why the write IOPS are so low? Aside from what Mark mentioned in his reply there's also latency to be considered in

Re: [ceph-users] OSD Performance

2015-02-24 Thread Mark Nelson
Hi Kevin, Writes are probably limited by a combination of locks, concurrent O_DSYNC journal writes, fsyncs, etc. The tests I mentioned were with both the OSD and the OSD journal on the same PCIe SSD. Others have looked into this in more detail than I have so might be able to chime in with

Re: [ceph-users] OSD Performance

2015-02-24 Thread Kevin Walker
Hi Mark Thanks for the info, 22k is not bad, but still massively below what a pcie ssd can achieve. Care to expand on why the write IOPS are so low? Was this with a separate RAM disk pcie device or SLC SSD for the journal? That fragmentation percentage looks good. We are considering using just

Re: [ceph-users] OSD Performance

2015-02-24 Thread Mark Nelson
On 02/24/2015 04:21 PM, Kevin Walker wrote: Hi All Just recently joined the list and have been reading/learning about ceph for the past few months. Overall it looks to be well suited to our cloud platform but I have stumbled across a few worrying items that hopefully you guys can clarify the sta

Re: [ceph-users] OSD Performance

2015-02-24 Thread Kevin Walker
Hi All Just recently joined the list and have been reading/learning about ceph for the past few months. Overall it looks to be well suited to our cloud platform but I have stumbled across a few worrying items that hopefully you guys can clarify the status of. Reading through various mailing list