On 22 aug 2011, at 23:28, Claudio Jeker wrote: > On Mon, Aug 22, 2011 at 10:49:47PM +0200, Per-Olov Sjvholm wrote: >> On 22 aug 2011, at 22:04, Stuart Henderson wrote: >>>> But if you can give hints of how to decrease the interrupt load I am all >> ears. >>>> As I see it, if the interrupt handling model i OpenBSD would change to a >>>> polling one u could maybe increase the throughput at the same processor >> speed >>>> (just me guessing though). But now the fact is that it is not polling. So >> what >>>> can I do with what we have.... >>> >>> polling is one mechanism to ensure you aren't handling interrupts all the >>> time, so you can ensure userland remains responsive even when the machine >> is >>> under heavy network load. OpenBSD has another way to handle this, MCLGETI. >> >> >> With polling if I get it right the context switch overhead is mostly avoided >> because the system can choose to look at the device when it is already in the >> right context. The drawback could be increased latency in processsing events >> in a polling model. But according to what I have read, the latency is reduced >> to a very low low values by raising the clock interrupt frequency. They say >> polling is better from a OS "time spent on device" control perspective. Note >> that I am not a pro in this area, but will for sure look deeper... > > Polling only works reliably at insane HZ settings which will cause other > issues at other places (some obvious some not so obvious). In the end > polling is a poor mans interrupt mitigation (which is also enabled on > em(4) btw.) since instead of using the interrupt of the network card you > use the interrupt of the clock to process the DMA rings. Polling does not > gain you much on good modern HW. > >> MCLGETI ?? Is it in if_em.c if I want to see how it is implemented? >> > > Yes. em(4) has MCLGETI(). > >>> >>>> Is pure cpu speed the only way? Or is it possible to decrease the >> interrupt >>>> load with even better NIC:s? >>> >>> here are some things that might help: >>> >>> - faster cpu >>> - larger cpu cache >>> - faster ram >>> - reduce overheads (things like switching VM context while handling >>> packets is not going to help matters) >>> - improving code efficiency >>> >>> have you tried -current? >>> >> >> >> >> I tried to share and use the same interrupt for my network ports as I have a >> guess it could be a boost, but the bios did not want what I wanted.... >> Interrupts could be shared, but not between the ports I wanted. I simple did >> not understand the interrupt allocation scheme in my Dell T410 tower server. >> >> Have not tried current, but will try current as soon as I can. Also... I will >> try to do some laborations with CPU speed of the core the OpenBSD virtual >> machine has. This to see how the interrupts and throughput is related to the >> CPU speed of the allocated core. >> > > Also make sure that the guest can actually access the physical HW directly > without any virtualisation in between. In the end real HW is going to have > less overhead and will be faster then a VM solution.
--snip-- The KVM hypervisor supports attaching PCI devices on the host system to virtualized guests. PCI passthrough allows guests to have exclusive access to PCI devices for a range of tasks. PCI passthrough allows PCI devices to appear and behave as if they were physically attached to the guest operating system. --snip-- From: http://docs.fedoraproject.org/en-US/Fedora/13/html/Virtualization_Guide/chap- Virtualization-PCI_passthrough.html The link above doesn't say anything about performance loss though of doing PCI pass through. But the OpenBSD indeed sees and uses the correct real physical NIC:s . I am of course _very_ interested in testing by installing OpenBSD directly on the hardware. But I cannot do that at this time. This is what the OpenBSD sees.. --snip-- em0 at pci0 dev 4 function 0 "Intel PRO/1000 MT (82574L)" rev 0x00: apic 1 int 11 (irq 11), address 00:1b:21:c2:8a:b0 em1 at pci0 dev 5 function 0 "Intel PRO/1000 MT (82574L)" rev 0x00: apic 1 int 10 (irq 10), address 00:1b:21:bf:76:77 --snip-- The MAC:s are these adapters real MAC:s. When used in OpenBSD these adapters are totally unbound in Linux and cannot be seen or used. This virtual fully patched OpenBSD 4.9 has got one (of total eight) Xeon 5504 2Ghz core, 512MB RAM and the above NIC:s and some raised values in sysctl. It (as said earlier) gives about max 400Mbit throughput with a small ruleset will keep state everywhere. Have tested with NFS, AFP, SCP, SMB and with different created 2GB ISO:s. All protocols gives near the same result (AFP performs best). Another physical server with a 1.6 Ghz Intel Atom with Intel Gig cards (not the same cards) performs similar (a little lower though) and max out at near the same speed. When these systems (both the physical and the virtual) max out, the interrupts eat 100%. Removing the firewall the file transfer give 119 Mbyte/s and max out the Gigabit pipe. These measurements (i.e comparison with the physical server) make me believe that the virtualization is not that bad. At least not from a performance perspective. A security discussion however is another topic. Maybe the best boost is a as Stuart said,... Faster CPU with more Ghz and faster memory. Or are there any other tweaks that can be looked into in OpenBSD? But after all, these are primarily some interesting and very fun tests. /Per-Olov
