Hi misc, unfortunately I'm facing a problem with my OBSD routers running 4.8-RELEASE
With top I can see very high interrupt cpu states, when forwarding high packet rates. Example: With nuttcp I generated traffic on systems connected to different router interfaces at the same time. It seems that the highest possible throughput is around 1.1 Gbps. CPU0 states: 0.0% user, 0.0% nice, 0.0% system, 94.8% interrupt, 5.2% idle CPU1 states: 0.0% user, 0.0% nice, 0.0% system, 0.0% interrupt, 100% idle Memory: Real: 10M/60M act/tot Free: 1911M Swap: 0K/4095M used/tot The hardware we're using are Network appliances barebone from Axiomtek, NA-820 (http://axiomtek.com/products/ViewProduct.asp?view=429) that has 7 onboard Intel NICs 82573L. In the obsd man page for the em(4) driver I found that the mentioned chipset does not provide VLAN tag insertion and stripping as well as jumbo frames. But that doesn't seem to be the problem as using VLAN tagging doesn't change the interruot states much. The same with disabling pf and carp. As those appliances are very cheap I first thought that 1.1 Gbps trhoughput might be the maximum in any case. With a different OS on two of the boxes I get at least 2 Gbps with cpus 96% idle. But as I need pf, pflow, carp, ospfd (in future) this is not an option. In the list archives I found a case with high cpu interrups using em but if I got it right it was related to ALTQ - and unfortunately unsolved. Any suggestions? Please find the dmesg output below. Best regards and thanks in advance, Bernd OpenBSD 4.8 (GENERIC.MP) #335: Mon Aug 16 09:09:20 MDT 2010 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 2136866816 (2037MB) avail mem = 2066161664 (1970MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.2 @ 0xf0800 (39 entries) bios0: vendor Phoenix Technologies, LTD version "6.00 PG" date 04/30/2010 acpi0 at bios0: rev 0 acpi0: sleep states S0 S1 S4 S5 acpi0: tables DSDT FACP MCFG APIC acpi0: wakeup devices PEX0(S5) PEX1(S5) PEX2(S5) PEX3(S5) PEX4(S5) PEX5(S5) HUB0(S5) UAR1(S5) UAR2(S5) USB0(S1) USB1(S1) USB2(S1) USB3(S1) USBE(S1) AC97(S5) AZAL(S5) PCI0(S5) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Pentium(R) Dual CPU E2220 @ 2.40GHz, 2400.38 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36, CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,EST,TM2 ,SSSE3,CX16,xTPR,PDCM,NXE,LONG cpu0: 1MB 64b/line 4-way L2 cache cpu0: apic clock running at 200MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Pentium(R) Dual CPU E2220 @ 2.40GHz, 2400.09 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36, CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,EST,TM2 ,SSSE3,CX16,xTPR,PDCM,NXE,LONG cpu1: 1MB 64b/line 4-way L2 cache ioapic0 at mainbus0: apid 4 pa 0xfec00000, version 20, 24 pins ioapic0: misconfigured as apic 0, remapped to apid 4 acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 2 (PEX0) acpiprt2 at acpi0: bus 3 (PEX1) acpiprt3 at acpi0: bus 4 (PEX2) acpiprt4 at acpi0: bus 5 (PEX3) acpiprt5 at acpi0: bus 6 (PEX4) acpiprt6 at acpi0: bus 7 (PEX5) acpiprt7 at acpi0: bus 8 (HUB0) acpicpu0 at acpi0 acpicpu1 at acpi0 acpitz0 at acpi0acpitz0: THRM: failed to read _TMP acpibtn0 at acpi0: PWRB pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 82945G Host" rev 0x02 ppb0 at pci0 dev 1 function 0 "Intel 82945G PCIE" rev 0x02: apic 4 int 16 (irq 7) pci1 at ppb0 bus 1 em0 at pci1 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 4 int 16 (irq 7), address 00:60:e0:4b:d2:a6 vga1 at pci0 dev 2 function 0 "Intel 82945G Video" rev 0x02 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) intagp0 at vga1 agp0 at intagp0: aperture at 0xc0000000, size 0x10000000 inteldrm0 at vga1: apic 4 int 16 (irq 7) drm0 at inteldrm0 ppb1 at pci0 dev 28 function 0 "Intel 82801GB PCIE" rev 0x01: apic 4 int 16 (irq 7) pci2 at ppb1 bus 2 em1 at pci2 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 4 int 16 (irq 7), address 00:60:e0:4b:d2:a7 ppb2 at pci0 dev 28 function 1 "Intel 82801GB PCIE" rev 0x01: apic 4 int 17 (irq 5) pci3 at ppb2 bus 3 em2 at pci3 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 4 int 17 (irq 5), address 00:60:e0:4b:d2:a8 ppb3 at pci0 dev 28 function 2 "Intel 82801GB PCIE" rev 0x01: apic 4 int 18 (irq 10) pci4 at ppb3 bus 4 em3 at pci4 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 4 int 18 (irq 10), address 00:60:e0:4b:d2:a9 ppb4 at pci0 dev 28 function 3 "Intel 82801GB PCIE" rev 0x01: apic 4 int 19 (irq 11) pci5 at ppb4 bus 5 em4 at pci5 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 4 int 19 (irq 11), address 00:60:e0:4b:d2:aa ppb5 at pci0 dev 28 function 4 "Intel 82801G PCIE" rev 0x01: apic 4 int 16 (irq 7) pci6 at ppb5 bus 6 em5 at pci6 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 4 int 16 (irq 7), address 00:60:e0:4b:d2:ab ppb6 at pci0 dev 28 function 5 "Intel 82801G PCIE" rev 0x01: apic 4 int 17 (irq 5) pci7 at ppb6 bus 7 em6 at pci7 dev 0 function 0 "Intel PRO/1000MT (82573L)" rev 0x00: apic 4 int 17 (irq 5), address 00:60:e0:4b:d2:ac uhci0 at pci0 dev 29 function 0 "Intel 82801GB USB" rev 0x01: apic 4 int 23 (irq 11) uhci1 at pci0 dev 29 function 1 "Intel 82801GB USB" rev 0x01: apic 4 int 19 (irq 11) uhci2 at pci0 dev 29 function 2 "Intel 82801GB USB" rev 0x01: apic 4 int 18 (irq 10) uhci3 at pci0 dev 29 function 3 "Intel 82801GB USB" rev 0x01: apic 4 int 16 (irq 7) ehci0 at pci0 dev 29 function 7 "Intel 82801GB USB" rev 0x01: apic 4 int 23 (irq 11) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb7 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xe1 pci8 at ppb7 bus 8 pcib0 at pci0 dev 31 function 0 "Intel 82801GB LPC" rev 0x01 pciide0 at pci0 dev 31 function 2 "Intel 82801GB SATA" rev 0x01: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI pciide0: using apic 4 int 19 (irq 0) for native-PCI interrupt wd0 at pciide0 channel 0 drive 1: <TOSHIBA MK2565GSX> wd0: 16-sector PIO, LBA48, 0MB, 0 sectors wd0(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 5 ichiic0 at pci0 dev 31 function 3 "Intel 82801GB SMBus" rev 0x01: apic 4 int 19 (irq 11) iic0 at ichiic0 spdmem0 at iic0 addr 0x50: 1GB DDR2 SDRAM non-parity PC2-6400CL5 spdmem1 at iic0 addr 0x52: 1GB DDR2 SDRAM non-parity PC2-6400CL5 usb1 at uhci0: USB revision 1.0 uhub1 at usb1 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb2 at uhci1: USB revision 1.0 uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb3 at uhci2: USB revision 1.0 uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb4 at uhci3: USB revision 1.0 uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo com0: console com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 wbsio0 at isa0 port 0x2e/2: W83627HF rev 0x41 lm1 at wbsio0 port 0x290/8: W83627HF mtrr: Pentium Pro MTRR support softraid0 at root root on wd0a swap on wd0b dump on wd0b carp50: state transition: BACKUP -> MASTER carp51: state transition: BACKUP -> MASTER carp52: state transition: BACKUP -> MASTER carp53: state transition: BACKUP -> MASTER carp54: state transition: BACKUP -> MASTER nd6_na_input: duplicate IP6 address fe80:000f::0200:5eff:fe00:0134 nd6_na_input: duplicate IP6 address fe80:0010::0200:5eff:fe00:0135 nd6_na_input: duplicate IP6 address fe80:0011::0200:5eff:fe00:0136
