On 2011-06-16 14:43, Peter Hallin wrote:
> On 2011-06-15 22:40, Jonathan Gray wrote:
> >
> > Yes this seems low indeed. You could try use rdomains and route exec
> > with a cable between the two ports to make packets go over the interfaces
> > if you don't have another host with 10G interfaces.
>
> I didn't get that far. When connecting the two ports to each other (tried
> with a couple of TP cables) I only get 1000baseT full-duplex on ix0 and
> ix1.
>
> >
> > If you've had ppb problems with em I'd imagine they would affect ix as well
> > if it is sitting behind the same ppb.
>
> I disabled the ppb hotplug interrupt in ppb.c as Claudio mentioned in
> "Performance degradation after upgrade" and ran the same tests again.
>
> This time it got up to 932 Mbit/s (close to theoretical maximum), but
> the CPU usage was 100% intrs. However, the interfaces only generated
> around 2000 intrs/s each (compared to 13000 in my last test).
>
> I will not have more time to do additional testing in a while as i'm off
> for vacation, but I hope my tests so far can give you some insight.
>
> Thanks, Peter
>
Hello, I finally got some time to test this 10G stuff again and now I'm using
two machines with the same cards running 5.0 RELEASE (amd64/sp).
Problem is as before, when connected and in UP state, the media goes
down to 1000BaseT. If I take them both down, it goes back up to
10GbaseT. Weird. Tested with different (brand new cables) both cat 5E
and cat 6.
Good news is that the interrupt rate seems to be better, only about 15% CPU
when running at 1Gbit/s and around 1500 intrs/s.
These are two identical machines with the same hardware.
//Peter
dmesg:
OpenBSD 5.0 (GENERIC) #53: Wed Aug 17 10:07:52 MDT 2011
dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC
RTC BIOS diagnostic error 80<clock_battery>
real mem = 3208368128 (3059MB)
avail mem = 3108941824 (2964MB)
mainbus0 at root
bios0 at mainbus0: SMBIOS rev. 2.5 @ 0xe4380 (23 entries)
bios0: vendor Intel Corp. version "BPX5810J.86B.0034.2010.0414.1511" date
04/14/2010
bios0: Intel Corporation WX58BP
acpi0 at bios0: rev 2
acpi0: sleep states S0 S1 S3 S4 S5
acpi0: tables DSDT FACP APIC WDDT MCFG ASF! HPET ASPT WDTx SSDT DMAR
acpi0: wakeup devices SLPB(S4) P32_(S4) ECIR(S4) UAR1(S4) ILAN(S4) PEG1(S4)
PEG3(S4) PEG7(S4) PEX0(S5) PEX1(S5) PEX2(S5) PEX3(S5) PEX4(S5) PEX5(S5)
UHC1(S3) UHC2(S3) UHC3(S3) UHC4(S3) EHCI(S3) EHC2(S3) UH42(S3) UHC5(S3)
UHC6(S3) AZAL(S3)
acpitimer0 at acpi0: 3579545 Hz, 24 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: Intel(R) Xeon(R) CPU W3690 @ 3.47GHz, 3467.31 MHz
cpu0:
FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,PCLMUL,MWAIT,DS-CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,DCA,SSE4.1,SSE4.2,POPCNT,AES,NXE,LONG
cpu0: 256KB 64b/line 8-way L2 cache
cpu0: apic clock running at 133MHz
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 20, 24 pins
acpimadt0: unknown apic structure type 7f
acpimcfg0 at acpi0 addr 0xf8000000, bus 0-63
acpihpet0 at acpi0: 14318179 Hz
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt1 at acpi0: bus 10 (P32_)
acpiprt2 at acpi0: bus 1 (PEG1)
acpiprt3 at acpi0: bus 5 (PEG3)
acpiprt4 at acpi0: bus 6 (PEG7)
acpiprt5 at acpi0: bus 7 (PEX0)
acpiprt6 at acpi0: bus 8 (PEX1)
acpiprt7 at acpi0: bus -1 (PEX3)
acpiprt8 at acpi0: bus 9 (PEX4)
acpicpu0 at acpi0: C1, PSS
acpibtn0 at acpi0: SLPB
cpu0: Enhanced SpeedStep 3467 MHz: speeds: 3459, 3458, 3325, 3192, 3059, 2926,
2793, 2660, 2527, 2394, 2261, 2128, 1995, 1862, 1729, 1596 MHz
pci0 at mainbus0 bus 0
pchb0 at pci0 dev 0 function 0 "Intel X58 Host" rev 0x13
ppb0 at pci0 dev 1 function 0 "Intel X58 PCIE" rev 0x13: msi
pci1 at ppb0 bus 1
ppb1 at pci1 dev 0 function 0 "IDT 89HPES12N3A" rev 0x0c
pci2 at ppb1 bus 2
ppb2 at pci2 dev 2 function 0 "IDT 89HPES12N3A" rev 0x0c
pci3 at ppb2 bus 3
em0 at pci3 dev 0 function 0 "Intel PRO/1000 QP (82576)" rev 0x01: msi, address
00:1b:21:c8:8f:ac
em1 at pci3 dev 0 function 1 "Intel PRO/1000 QP (82576)" rev 0x01: msi, address
00:1b:21:c8:8f:ad
ppb3 at pci2 dev 4 function 0 "IDT 89HPES12N3A" rev 0x0c
pci4 at ppb3 bus 4
em2 at pci4 dev 0 function 0 "Intel PRO/1000 QP (82576)" rev 0x01: msi, address
00:1b:21:c8:8f:ae
em3 at pci4 dev 0 function 1 "Intel PRO/1000 QP (82576)" rev 0x01: msi, address
00:1b:21:c8:8f:af
ppb4 at pci0 dev 3 function 0 "Intel X58 PCIE" rev 0x13: msi
pci5 at ppb4 bus 5
vga1 at pci5 dev 0 function 0 "NVIDIA GeForce 210" rev 0xa2
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
azalia0 at pci5 dev 0 function 1 vendor "NVIDIA", unknown product 0x0be3 rev
0xa1: msi
azalia0: no supported codecs
ppb5 at pci0 dev 7 function 0 "Intel X58 PCIE" rev 0x13: msi
pci6 at ppb5 bus 6
ix0 at pci6 dev 0 function 0 "Intel 82599 T3" rev 0x01: msi, address
00:1b:21:93:a6:40
ix1 at pci6 dev 0 function 1 "Intel 82599 T3" rev 0x01: msi, address
00:1b:21:93:a6:41
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 0 not configured
"Intel X58 QuickPath" rev 0x13 at pci0 dev 16 function 1 not configured
"Intel X58 Misc" rev 0x13 at pci0 dev 20 function 0 not configured
"Intel X58 GPIO" rev 0x13 at pci0 dev 20 function 1 not configured
"Intel X58 RAS" rev 0x13 at pci0 dev 20 function 2 not configured
"Intel X58 Throttle" rev 0x13 at pci0 dev 20 function 3 not configured
em4 at pci0 dev 25 function 0 "Intel ICH10 R BM LM" rev 0x00: msi, address
e0:69:95:69:aa:c7
uhci0 at pci0 dev 26 function 0 "Intel 82801JI USB" rev 0x00: apic 8 int 16
uhci1 at pci0 dev 26 function 1 "Intel 82801JI USB" rev 0x00: apic 8 int 21
uhci2 at pci0 dev 26 function 2 "Intel 82801JI USB" rev 0x00: apic 8 int 19
ehci0 at pci0 dev 26 function 7 "Intel 82801JI USB" rev 0x00: apic 8 int 18
usb0 at ehci0: USB revision 2.0
uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb6 at pci0 dev 28 function 0 "Intel 82801JI PCIE" rev 0x00
pci7 at ppb6 bus 7
ppb7 at pci0 dev 28 function 1 "Intel 82801JI PCIE" rev 0x00
pci8 at ppb7 bus 8
ppb8 at pci0 dev 28 function 4 "Intel 82801JI PCIE" rev 0x00
pci9 at ppb8 bus 9
uhci3 at pci0 dev 29 function 0 "Intel 82801JI USB" rev 0x00: apic 8 int 23
uhci4 at pci0 dev 29 function 1 "Intel 82801JI USB" rev 0x00: apic 8 int 19
uhci5 at pci0 dev 29 function 2 "Intel 82801JI USB" rev 0x00: apic 8 int 18
ehci1 at pci0 dev 29 function 7 "Intel 82801JI USB" rev 0x00: apic 8 int 23
usb1 at ehci1: USB revision 2.0
uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1
ppb9 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0x90
pci10 at ppb9 bus 10
"TI TSB43AB22 FireWire" rev 0x00 at pci10 dev 3 function 0 not configured
pcib0 at pci0 dev 31 function 0 "Intel 82801JIR LPC" rev 0x00
pciide0 at pci0 dev 31 function 2 "Intel 82801JI SATA" rev 0x00: DMA, channel 0
configured to native-PCI, channel 1 configured to native-PCI
pciide0: using apic 8 int 19 for native-PCI interrupt
wd0 at pciide0 channel 0 drive 0: <WDC WD5000AADS-00M2B0>
wd0: 16-sector PIO, LBA48, 476940MB, 976773168 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 6
ichiic0 at pci0 dev 31 function 3 "Intel 82801JI SMBus" rev 0x00: apic 8 int 18
iic0 at ichiic0
iic0: addr 0x2e 1a=80 1b=80 1c=80 1d=04 1e=87 1f=10 20=56 21=6a 22=be 23=c5
24=c6 25=36 26=26 27=29 28=10 29=0c 2a=ff 2b=ff 2c=ff 2d=ff 2e=ff 2f=ff 30=33
31=80 32=80 33=bc 34=81 36=c5 38=ff 39=ff 3a=ff 3b=e2 3c=7a 3d=ef 3e=41 3f=6f
40=35 45=ff 47=ff 49=ff 4b=ff 4d=ff 4e=81 4f=7f 50=81 51=7f 52=81 53=7f 54=ff
55=ff 56=ff 57=ff 58=ff 59=ff 5a=ff 5b=ff 5c=0a 5d=22 5e=22 5f=7a 60=8a 61=8a
62=e8 63=aa 64=33 65=80 66=80 67=55 68=2d 69=2d 6a=67 6b=41 6c=41 6d=04 6e=42
76=ed 77=d5 78=f0 7b=55 7c=01 86=ff 87=ff 8a=fb 8b=64 8c=64 8d=64 91=5d 92=5d
93=5d words 00=00e5 01=00e5 02=00e5 03=00e5 04=00e5 05=00e5 06=00e5 07=00e5
spdmem0 at iic0 addr 0x50: 1GB DDR3 SDRAM PC3-10600
spdmem1 at iic0 addr 0x52: 1GB DDR3 SDRAM PC3-10600
spdmem2 at iic0 addr 0x54: 1GB DDR3 SDRAM PC3-10600
pciide1 at pci0 dev 31 function 5 "Intel 82801JI SATA" rev 0x00: DMA, channel 0
wired to native-PCI, channel 1 wired to native-PCI
pciide1: using apic 8 int 18 for native-PCI interrupt
atapiscsi0 at pciide1 channel 1 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <TSSTcorp, CDDVDW SH-222AB, SB01> ATAPI 5/cdrom
removable
cd0(pciide1:1:0): using PIO mode 4, Ultra-DMA mode 5
usb2 at uhci0: USB revision 1.0
uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb3 at uhci1: USB revision 1.0
uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb4 at uhci2: USB revision 1.0
uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb5 at uhci3: USB revision 1.0
uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb6 at uhci4: USB revision 1.0
uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1
usb7 at uhci5: USB revision 1.0
uhub7 at usb7 "Intel UHCI root hub" rev 1.00/1.00 addr 1
isa0 at pcib0
isadma0 at isa0
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pcppi0 at isa0 port 0x61
spkr0 at pcppi0
mtrr: Pentium Pro MTRR support
uhidev0 at uhub7 port 2 configuration 1 interface 0 "CHICONY HP Basic USB
Keyboard" rev 1.10/3.00 addr 2
uhidev0: iclass 3/1
ukbd0 at uhidev0: 8 modifier keys, 6 key codes
wskbd1 at ukbd0 mux 1
wskbd1: connecting to wsdisplay0
vscsi0 at root
scsibus1 at vscsi0: 256 targets
softraid0 at root
scsibus2 at softraid0: 256 targets
root on wd0a (83390ad02a1f706a.a) swap on wd0b dump on wd0b
