We are days away from completing our production CloudStack / Ceph deployment but have run into a small challenge in regards to Ceph performance. As you run a setup similar to ours I was wondering if you've seen a similar issue?
Our Ceph cluster consists of 60 X 7.2K SAS drive, spread over 5 nodes each with a 960GB Samsung NVMe. Our KVM hypervisors are connected to this nodes via 2 x 10Gbps links, per node. 3 centos Cloudstack management servers 4.11.0 4 centos KVM hosts The RAW Ceph performance looks great! CEPH RAW Average IOPS Max IOPS MB/sec Write 4K 5,885 6,190 22.9907 Read 4K 28,985 35,025 113 Write 4MB 204 219 816 Read 4MB 361 399 1447.51 However when we run the tests from within KVM we get significantly reduced performance, particular in writes: VM Running on KVM Hypervisor Average IOPS Max IOPS MB/sec Write 4K 946.07 1390 3.69 Write 4MB 472 614 116 We believe the libvirt/ceph library could be causing the problem with in centos as we don't see this on a Ubuntu KVM HOST Kind Regards, [Oakford Internet Services]<http://www.oakfordis.com/> Adam Witwicki | Hosted Systems Specialist 01380 710278 / [email protected]<mailto:[email protected]> Oakford Internet Services Office: 01380 888088 10 Prince Maurice Court, Devizes, Wiltshire. SN10 2RT www.oakfordis.com<http://www.oakfordis.com/> [email protected]<mailto:[email protected]> Disclaimer Notice: This email has been sent by Oakford Technology Limited, while we have checked this e-mail and any attachments for viruses, we can not guarantee that they are virus-free. You must therefore take full responsibility for virus checking. This message and any attachments are confidential and should only be read by those to whom they are addressed. If you are not the intended recipient, please contact us, delete the message from your computer and destroy any copies. Any distribution or copying without our prior permission is prohibited. Internet communications are not always secure and therefore Oakford Technology Limited does not accept legal responsibility for this message. The recipient is responsible for verifying its authenticity before acting on the contents. Any views or opinions presented are solely those of the author and do not necessarily represent those of Oakford Technology Limited. Registered address: Oakford Technology Limited, 10 Prince Maurice Court, Devizes, Wiltshire. SN10 2RT. Registered in England and Wales No. 5971519
