Hi, I'm trying to tune the network speed of OpenBSD box for high bandwidht, and high latency. The box is connected to a 155MBit Internet uplink.
Hosts I have here next to me: An old OpenBSD 4.4 box, used as server firewall, in front of a Linux http server. A new OpenBSD 5.0 box, used as http server. A Linux OpenSUSE 11.2 box, used as http server. Further I have: A Linux based VM in Canada, which I use as client. and using OpenBSD mirror artifiles.org as reference host. First tests with the OpenBSD 4.4 firewall: ping roundtrip times: from the Linux VM in Canada to artfiles.org: about time=150 ms from the Linux VM in Canada to my server/firewall: about time=155 ms from my server to artifles.org: about 8/9 ms Downloading src.tar.gz from the reference box artfiles.org from the Linux VM in Canada: I get about 7.9 MB/s Downloading a file on the Linux VM from the Linux server behind the 4.4 Firewall I get about 2MB/s. While downloading the src.tar.gz, the speed goes a bit up and down, i.e. up to 3.5 MB, and down to 700Kb. I asked already a couple of years ago, and got an answer about the Bandwith * Delay product from Otto. I calculated what to use for the tcp send and receive buffers, but whatever I put there, it doesn't seem to matter much. Speed stays at around 2MB/s. Trying to download the file multiple times, then the speed of about 2MB/s is divided to the amount of streams. Then I took the OpenSUSE 11.2 box, and put it on the Internet link. On that box, I downloaded again the src.tar.gz from artfiles.org, and got about 15/16 MB/s. So more or less the full 155MBit. I installed apache2 on that box, and put the src.tar.gz in the htdocs, and downloaded from the Linux VM in Canada. There I got download speeds of average 2.6 MB/s. Then I put a second harddisk in the PC and installled OpenBSD 5.0 amd64 on it. Trying to download src.tar.gz from artfiles.org I got around 10MB/s, so about 5/6 MB/s fewer than with the Linux host. Looking a bit closer on the ifconfig em0 output, I recognized, its running with 100MBit full duplex speed, but the box is connected to a Gigabit switch, and can handle GigaBit. Under Linux it auto-negotiates 100MB/s full duplex. So at least that explains me here, why I don't get the full 155MBit/s, but still wondering, why the card doesn't support/use gigabit. man em0 doesn't seem to mention the card I have, but also only states that the 82562V chipset only supports 10/100MBit. A dmesg is attached. Then I started apache, and put the src.tar.gz in its htdocs directory. Downloading this src.tar.gz from the Linux VM in Canada, I get 1.45MB/s. But this speed seems to be constant. When I downloaded from the Canadian VM from the OpenSUSE 11.2 Linux host, or from the Linxu server behind the OpenBSD 4.4 Firewall, the speed was not stable, going up and down between 0.5 MBit and 3.X MBit. But downloading directly from the 5.0 host, the speed goes up to 1.5MB/s, and then more or less stays constantly at this speed. In 5.0, the net.inet.tcp.recvspace, and net.inet.tcp.sendspace seem to be gone. I was looking at https://calomel.org/network_performance.html what I can potentially do to speed up connection, but if I understand that right, this article mostly focuses on speed up local LAN connections. Anyways, I tried to raise the kern.bufcachepercent, and the net.inet.tcl.mssdflt, but without noticeable difference. In the FAQ I also don't find much help: http://openbsd.das.ufsc.br/faq/faq6.html#Tuning So, why I don't get the 155MBit I know when downloading from a near server, is kind of understandable for me, but I still wonder whether there is a way to get at least the same speed when downloading on the Canadian Linux VM, from the OpenBSD 5.0 host here in Germany, so getting 2.6 MB/s instead of 1.45MB/s. If someone could point me to the right RTFM, or has a hint at hand what might be worth to try to tweak, please let me know. Sebastian OpenBSD 5.0 (GENERIC.MP) #63: Wed Aug 17 10:14:30 MDT 2011 dera...@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP RTC BIOS diagnostic error 80<clock_battery> real mem = 3209342976 (3060MB) avail mem = 3109855232 (2965MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xe3d70 (36 entries) bios0: vendor Intel Corp. version "CO96510J.86A.5882.2007.0413.0100" date 04/13/2007 bios0: wortmann DQ965CO acpi0 at bios0: rev 0 acpi0: sleep states S0 S3 S4 S5 acpi0: tables DSDT FACP APIC WDDT MCFG ASF! SSDT SSDT SSDT SSDT SSDT TCPA acpi0: wakeup devices SLPB(S4) P32_(S4) UAR1(S4) UAR2(S4) ILAN(S4) PEGP(S4) PEX0(S4) PEX1(S4) PEX2(S4) PEX3(S4) PEX4(S4) PEX5(S4) UHC1(S3) UHC2(S3) UHC3(S3) UHC4(S3) EHCI(S3) EHC2(S3) UH42(S3) UHC5(S3) AZAL(S3) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz, 1866.97 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG cpu0: 2MB 64b/line 8-way L2 cache cpu0: apic clock running at 266MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM)2 CPU 6300 @ 1.86GHz, 1866.73 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,NXE,LONG cpu1: 2MB 64b/line 8-way L2 cache ioapic0 at mainbus0: apid 2 pa 0xfec00000, version 20, 24 pins ioapic0: misconfigured as apic 0, remapped to apid 2 acpimcfg0 at acpi0 addr 0xf0000000, bus 0-127 acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 7 (P32_) acpiprt2 at acpi0: bus 2 (PEX0) acpiprt3 at acpi0: bus 3 (PEX1) acpiprt4 at acpi0: bus 4 (PEX2) acpiprt5 at acpi0: bus 5 (PEX3) acpiprt6 at acpi0: bus 6 (PEX4) acpiprt7 at acpi0: bus -1 (PEX5) acpicpu0 at acpi0: PSS acpicpu1 at acpi0: PSS acpibtn0 at acpi0: SLPB cpu0: Enhanced SpeedStep 1866 MHz: speeds: 1862, 1596 MHz pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 "Intel 82Q965 Host" rev 0x02 ppb0 at pci0 dev 1 function 0 "Intel 82Q965 PCIE" rev 0x02: msi pci1 at ppb0 bus 1 vga1 at pci1 dev 0 function 0 "ATI Radeon X1300/X1550" rev 0x00 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) radeondrm0 at vga1: apic 2 int 16 drm0 at radeondrm0 "ATI Radeon X1300/X1550 Sec" rev 0x00 at pci1 dev 0 function 1 not configured "Intel 82Q965 HECI" rev 0x02 at pci0 dev 3 function 0 not configured pciide0 at pci0 dev 3 function 2 "Intel 82Q965 PT IDER" rev 0x02: DMA (unsupported), channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide0: using apic 2 int 18 for native-PCI interrupt pciide0: channel 0 ignored (not responding; disabled or no drives?) pciide0: channel 1 ignored (not responding; disabled or no drives?) "Intel 82Q965 KT" rev 0x02 at pci0 dev 3 function 3 not configured em0 at pci0 dev 25 function 0 "Intel ICH8 IGP AMT" rev 0x02: msi, address 00:19:d1:72:eb:f1 uhci0 at pci0 dev 26 function 0 "Intel 82801H USB" rev 0x02: apic 2 int 16 uhci1 at pci0 dev 26 function 1 "Intel 82801H USB" rev 0x02: apic 2 int 21 ehci0 at pci0 dev 26 function 7 "Intel 82801H USB" rev 0x02: apic 2 int 18 usb0 at ehci0: USB revision 2.0 uhub0 at usb0 "Intel EHCI root hub" rev 2.00/1.00 addr 1 azalia0 at pci0 dev 27 function 0 "Intel 82801H HD Audio" rev 0x02: msi azalia0: codecs: Sigmatel STAC9227X audio0 at azalia0 ppb1 at pci0 dev 28 function 0 "Intel 82801H PCIE" rev 0x02: msi pci2 at ppb1 bus 2 ppb2 at pci0 dev 28 function 1 "Intel 82801H PCIE" rev 0x02: msi pci3 at ppb2 bus 3 pciide1 at pci3 dev 0 function 0 "Marvell 88SE6101 IDE" rev 0xb1: DMA (unsupported), channel 0 configured to native-PCI, channel 1 configured to native-PCI pciide1: using apic 2 int 17 for native-PCI interrupt pciide1: channel 0 ignored (not responding; disabled or no drives?) pciide1: channel 1 ignored (not responding; disabled or no drives?) ppb3 at pci0 dev 28 function 2 "Intel 82801H PCIE" rev 0x02: msi pci4 at ppb3 bus 4 ppb4 at pci0 dev 28 function 3 "Intel 82801H PCIE" rev 0x02: msi pci5 at ppb4 bus 5 ppb5 at pci0 dev 28 function 4 "Intel 82801H PCIE" rev 0x02: msi pci6 at ppb5 bus 6 uhci2 at pci0 dev 29 function 0 "Intel 82801H USB" rev 0x02: apic 2 int 23 uhci3 at pci0 dev 29 function 1 "Intel 82801H USB" rev 0x02: apic 2 int 19 uhci4 at pci0 dev 29 function 2 "Intel 82801H USB" rev 0x02: apic 2 int 18 ehci1 at pci0 dev 29 function 7 "Intel 82801H USB" rev 0x02: apic 2 int 23 usb1 at ehci1: USB revision 2.0 uhub1 at usb1 "Intel EHCI root hub" rev 2.00/1.00 addr 1 ppb6 at pci0 dev 30 function 0 "Intel 82801BA Hub-to-PCI" rev 0xf2 pci7 at ppb6 bus 7 "TI TSB43AB22 FireWire" rev 0x00 at pci7 dev 3 function 0 not configured pcib0 at pci0 dev 31 function 0 "Intel 82801HO LPC" rev 0x02 pciide2 at pci0 dev 31 function 2 "Intel 82801H SATA" rev 0x02: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI pciide2: using apic 2 int 19 for native-PCI interrupt wd0 at pciide2 channel 0 drive 0: <WDC WD800JD-00LSA0> wd0: 16-sector PIO, LBA48, 76319MB, 156301488 sectors wd0(pciide2:0:0): using PIO mode 4, Ultra-DMA mode 6 ichiic0 at pci0 dev 31 function 3 "Intel 82801H SMBus" rev 0x02: apic 2 int 21 iic0 at ichiic0 spdmem0 at iic0 addr 0x50: 1GB DDR2 SDRAM non-parity PC2-4200CL5 spdmem1 at iic0 addr 0x51: 1GB DDR2 SDRAM non-parity PC2-6400CL5 spdmem2 at iic0 addr 0x52: 1GB DDR2 SDRAM non-parity PC2-4200CL5 pciide3 at pci0 dev 31 function 5 "Intel 82801H SATA" rev 0x02: DMA, channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide3: using apic 2 int 19 for native-PCI interrupt usb2 at uhci0: USB revision 1.0 uhub2 at usb2 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb3 at uhci1: USB revision 1.0 uhub3 at usb3 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb4 at uhci2: USB revision 1.0 uhub4 at usb4 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb5 at uhci3: USB revision 1.0 uhub5 at usb5 "Intel UHCI root hub" rev 1.00/1.00 addr 1 usb6 at uhci4: USB revision 1.0 uhub6 at usb6 "Intel UHCI root hub" rev 1.00/1.00 addr 1 isa0 at pcib0 isadma0 at isa0 com0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo pckbc0 at isa0 port 0x60/5 pckbd0 at pckbc0 (kbd slot) pckbc0: using irq 1 for kbd slot wskbd0 at pckbd0: console keyboard, using wsdisplay0 pcppi0 at isa0 port 0x61 spkr0 at pcppi0 mtrr: Pentium Pro MTRR support uhidev0 at uhub4 port 1 configuration 1 interface 0 "BTC USB Multimedia Keyboard" rev 1.10/1.00 addr 2 uhidev0: iclass 3/1 ukbd0 at uhidev0: 8 modifier keys, 6 key codes wskbd1 at ukbd0 mux 1 wskbd1: connecting to wsdisplay0 uhidev1 at uhub4 port 1 configuration 1 interface 1 "BTC USB Multimedia Keyboard" rev 1.10/1.00 addr 2 uhidev1: iclass 3/0, 3 report ids uhid0 at uhidev1 reportid 1: input=1, output=0, feature=0 uhid1 at uhidev1 reportid 2: input=3, output=0, feature=0 uhid2 at uhidev1 reportid 3: input=3, output=0, feature=8 vscsi0 at root scsibus0 at vscsi0: 256 targets softraid0 at root scsibus1 at softraid0: 256 targets root on wd0a (239ab09ca538f21a.a) swap on wd0b dump on wd0b umass0 at uhub1 port 4 configuration 1 interface 0 "Generic Mass Storage" rev 2.00/1.03 addr 2 umass0: using SCSI over Bulk-Only scsibus2 at umass0: 2 targets, initiator 0 sd0 at scsibus2 targ 1 lun 0: <Generic, Flash Disk, 8.07> SCSI2 0/direct removable sd0: 7712MB, 512 bytes/sector, 15794176 sectors
