Maybe some tracing on kvm process could give us clues to find where the cpu is used ?
Also another idea, can you try with "auth supported=none" ? maybe they are some overhead with ceph authenfication ? ----- Mail original ----- De: "Alexandre DERUMIER" <[email protected]> À: "Stefan Priebe - Profihost AG" <[email protected]> Cc: [email protected] Envoyé: Lundi 12 Novembre 2012 15:20:07 Objet: Re: [pve-devel] less cores more iops / speed Ok thanks. Seem to use a lot of cpu vs nfs,iscsi ... I hope that ceph dev will work on this soon ! ----- Mail original ----- De: "Stefan Priebe - Profihost AG" <[email protected]> À: "Alexandre DERUMIER" <[email protected]> Cc: "eric" <[email protected]>, [email protected] Envoyé: Lundi 12 Novembre 2012 15:05:08 Objet: Re: [pve-devel] less cores more iops / speed Am 12.11.2012 13:49, schrieb Alexandre DERUMIER: >>> One VM on one Host: 18.000 IOP/s >>> Two VM on one Host: 2x11.000 IOP/s >>> Three VM on one Host: 3x7.000 IOP/s > > And host cpu is 100% ? No. For three VMs yes. For one and two no. I think librbd / rbd implementation in kvm is the bottleneck here. Stefan > ----- Mail original ----- > > De: "Stefan Priebe - Profihost AG" <[email protected]> > À: "Alexandre DERUMIER" <[email protected]> > Cc: "eric" <[email protected]>, [email protected] > Envoyé: Lundi 12 Novembre 2012 12:58:35 > Objet: Re: [pve-devel] less cores more iops / speed > > Am 12.11.2012 08:51, schrieb Alexandre DERUMIER: >>>> Right now RBD in KVM is limited by CPU speed. >> >> Good to known, so it's seem lack of threading, or maybe somes locks. (so >> faster cpu give more iops). >> >> If you lauch parallel fio on same host on different guest, do you get more >> total iops ? (for me it's scale) > > One VM on one Host: 18.000 IOP/s > Two VM on one Host: 2x11.000 IOP/s > Three VM on one Host: 3x7.000 IOP/s > >> if you launch 2 parallel fio, on same guest (on differents disk), do you get >> more iops ? (for me, it doesn't scale, so raid0 in guest doesn't help). > No it doesn't scale. > > Stefan > >> ----- Mail original ----- >> >> De: "Stefan Priebe" <[email protected]> >> À: "Alexandre DERUMIER" <[email protected]> >> Cc: "eric" <[email protected]>, [email protected] >> Envoyé: Dimanche 11 Novembre 2012 13:07:36 >> Objet: Re: [pve-devel] less cores more iops / speed >> >> Am 11.11.2012 12:12, schrieb Alexandre DERUMIER: >>> If I remember good, stefan can achieve 100.000 iops with iscsi with same >>> kvm host. >> >> Correct but this was always with scsi-generic and I/O multipathing on >> host. rbd does not support scsi-generic ;-( >> >>> I have checked ceph mailing, stefan seem to have resolved his problem with >>> dual core with bios update ! >> Correct. So speed on Dual Xeon is now 14.000 IOP/s and 18.000 IOP/s on >> Single Xeon. But the difference is an issue of the CPU Speed. 3,6Ghz >> Single Xeon vs. 2.5Ghz Dual Xeon. >> >> Right now RBD in KVM is limited by CPU speed. >> >> Greets, >> Stefan >> _______________________________________________ pve-devel mailing list [email protected] http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
_______________________________________________ pve-devel mailing list [email protected] http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
