Hi Chris, I'm doing something very similar to you, however only in a very early stage of testing, but I don't seem to see the same problem that you are experiencing.
My setup is as follows:- 1x HP DL360 Server Running ESX 5.1 8x 10K SAS 146GB drives each configured as a RAID 0 and with a separate ESX Datastore on them 4 Virtual Ceph VM's with 2 OSD's each residing on above datastores Ceph VM's running Ubuntu 14.04 with manually upgraded kernel to 3.16 2x Replicated pool over all OSD's with a test 50GB RBD The first 3 VM's also function as a Pacemaker cluster which exports a HA LIO iSCSI target ESX server then remounts this iSCSI target as a new datastore Windows 2008 VM running IOmeter sits on this datastore I realise this is a bit of a mess, but it has allowed me to do some basic testing and understand the theory behind gluing Ceph, LIO and pacemaker together. The next stage will be to try and source some more production like hardware to do a more lifelike test. But back to the original point, from this limited hardware I am seeing 200-400MB/s 256kb reads in IOMeter, which is probably about right for the number of disks. I'm afraid I can't offer much advice at this stage as I'm still fairly early on in understanding Ceph/LIO, but maybe that my performance is not limited suggests that in some combination what we are both trying to achieve is possible. I assume you are using the kernel RBD client? Only other things I can think to try from basic fault finding experience, would be trying the same Kernel version and/or Ubuntu in case there are some minor differences which cause this problem. Of course it maybe that the nested nature of my setup is masking this problem. If you need any configs from my test cluster to compare or any specific tests runs, please let me know. Nick Nick Fisk Technical Support Engineer System Professional Ltd tel: 01825 830000 mob: 07711377522 fax: 01825 830001 mail: nick.f...@sys-pro.co.uk web: www.sys-pro.co.uk<http://www.sys-pro.co.uk> IT SUPPORT SERVICES | VIRTUALISATION | STORAGE | BACKUP AND DR | IT CONSULTING Registered Office: Wilderness Barns, Wilderness Lane, Hadlow Down, East Sussex, TN22 4HU Registered in England and Wales. Company Number: 04754200 Confidentiality: This e-mail and its attachments are intended for the above named only and may be confidential. If they have come to you in error you must take no action based on them, nor must you copy or show them to anyone; please reply to this e-mail and highlight the error. Security Warning: Please note that this e-mail has been created in the knowledge that Internet e-mail is not a 100% secure communications medium. We advise that you understand and observe this lack of security when e-mailing us. Viruses: Although we have taken steps to ensure that this e-mail and attachments are free from any virus, we advise that in keeping with good computing practice the recipient should ensure they are actually virus free. Any views expressed in this e-mail message are those of the individual and not necessarily those of the company or any of its subsidiaries.
_______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
