Not using rbd, just lvm on the local array. Just wanted to point out that a single vm, regardless of storage used, does seem to be limited on IOPS.
I suspect the cpu time needed to handle each io operation in KVM/qemu is where the limit comes from. When I first started testing I got lower results. Then I set some options to get turbo boost to kick in for all cores all the time resulting in better IOPS. I was testing using crystal diskmark in windows using virtio. Alexandre DERUMIER <[email protected]> wrote:Hi Eric, >>I have been testing IOPS with Areca 1882 pcie 3.0 with SSD disks. >>A single VM on my system seems to be limited to about 30K IOPS writing >>and about 70K reading. >>I can get more total IOPS running benchmarks on two VMs at the same time. Great Results ! What benchmark do you use ? fio ? can you post the bench command line to compare ? Also what is your cluster hardware (cpu,ram,disk model, how many nodes, switchs models ?). Your results seem to be a lot higher than stephan. (This is the best results I ever see with rbd ;) >>Instructions here: >>http://forum.proxmox.com/threads/11508-Intel-Turbo-Boost-not-working?p=63058#post63058 I agree with this settings, I always set this options in bios. Also maybe irqbalance daemon can help: #apt-get install irqbalance ----- Mail original ----- De: "Eric Blevins" <[email protected]> À: [email protected] Envoyé: Vendredi 9 Novembre 2012 21:53:21 Objet: Re: [pve-devel] less cores more iops / speed I have been testing IOPS with Areca 1882 pcie 3.0 with SSD disks. A single VM on my system seems to be limited to about 30K IOPS writing and about 70K reading. I can get more total IOPS running benchmarks on two VMs at the same time. Xeon E5-2650 2.0Ghz 8 core These newer Xeon with turboboost make it a little difficult to benchmark if frequencies change all the time. With a few teaks I got all the cores in my 2650s to run at 2.4Ghz all the time. Instructions here: http://forum.proxmox.com/threads/11508-Intel-Turbo-Boost-not-working?p=63058#post63058 On 11/09/2012 11:02 AM, Alexandre DERUMIER wrote: > Oh great, > so it seem that you can get more ios with more vm. (It's become interesting > for me :) > > What are you exact xeon models ? same server model,bios options ? > > what happen on the dual xeon, if you shutdown some cores ? > > echo 0 > /sys/devices/system/cpu/cpuX/online > > > also, does pining help on dual xeon ? > > > > ----- Mail original ----- > > De: "Stefan Priebe - Profihost AG" <[email protected]> > À: "Alexandre DERUMIER" <[email protected]> > Cc: [email protected] > Envoyé: Vendredi 9 Novembre 2012 16:42:09 > Objet: Re: [pve-devel] less cores more iops / speed > > Am 09.11.2012 15:49, schrieb Alexandre DERUMIER: >> Maybe one last question (sorry ;) >> >> Do you have tried to launch 2 fio test from 2 differents kvm host at the >> same time ? >> >> If yes, do you have more io or do you have half of speed on each fio test ? > Only one VM active on a Single Xeon 3.6Ghz: > write: io=6024MB, bw=68519KB/s, iops=17129, runt= 90025msec > read : io=6057MB, bw=68891KB/s, iops=17222, runt= 90026msec > write: io=71540MB, bw=808308KB/s, iops=197, runt= 90630msec > read : io=147096MB, bw=1631MB/s, iops=407, runt= 90162msec > > > Only one VM active on a Dual Xeon 2.5Ghz: > write: io=2224MB, bw=25285KB/s, iops=6321, runt= 90070msec > read : io=2033MB, bw=23108KB/s, iops=5777, runt= 90078msec > write: io=50900MB, bw=574235KB/s, iops=140, runt= 90767msec > read : io=84228MB, bw=954499KB/s, iops=233, runt= 90361msec > > > Both VMs active (doing each test in parallel): > > VM 1: > write: io=5376MB, bw=61112KB/s, iops=15277, runt= 90086msec > read : io=6092MB, bw=69292KB/s, iops=17323, runt= 90028msec > write: io=67724MB, bw=766012KB/s, iops=187, runt= 90533msec > read : io=160200MB, bw=1776MB/s, iops=444, runt= 90187msec > > VM 2: > write: io=2351MB, bw=26719KB/s, iops=6679, runt= 90083msec > read : io=2190MB, bw=24899KB/s, iops=6224, runt= 90081msec > write: io=48320MB, bw=545652KB/s, iops=133, runt= 90680msec > read : io=88136MB, bw=999571KB/s, iops=244, runt= 90290msec > > Right now i've no idea why the Dual Xeon is slower at all / in general. > > Stefan > _______________________________________________ > pve-devel mailing list > [email protected] > http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel _______________________________________________ pve-devel mailing list [email protected] http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
_______________________________________________ pve-devel mailing list [email protected] http://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
