On 1/7/2012 4:48 AM, Stuart Henderson wrote:
In gmane.os.openbsd.misc, you wrote:
I'm trying to troubleshoot some performance issues for high speed data
transfers across a long network path with a fairly high bandwidth delay.
Any difference between TCP and UDP?
I haven't been able to compare fully since I don't have a udp iperf
server at the far end; mostly I used the udp iperf test for its
reporting of packet loss (it showed the same loss with or without going
through the firewall). Our first suspicion was packet drops at the
router, which would have a huge effect on the long path, but that
doesn't seem to be the case.
As a test to help pinpoint things, can you try passing the traffic
near the top of your ruleset with 'pass quick..flags any no state'?
Or with PF ddisabled if that's possible?
"no state" would be an interesting test - will try that and report. I'd
like to try with pf disabled but there are a few hundred machines behind
it which I don't want to open up. Maybe with a separate test rig...
Anything in syslog from PF? How about after pfctl -xmisc?
Haven't seen anything but I will switch on the "misc" logging (that's a
setting I missed!).
Is this path using the same network interface as you've used in
local tests?
Yes, it has only 3 physical interfaces (internal/external/pfsync), it
does have a bunch of vlan and carp interfaces defined on the internal
physical interface but everything is the same for local/remote tests.
Always worth including dmesg, irq assignments might be interesting.
I will include that now... I know it's always useful but my initial
message seemed pretty long already. It's a Dell 1750; I disabled any
non-essential devices in the BIOS (USB, floppy, serial etc) though there
are still a few extras showing up from the DRAC module...
Are any interfaces marked 'down'?
Are you using pfsync?
Doing any bridging or just routing?
Nothing marked 'down' though the second onboard NIC bge1 is unused. It
uses pfsync on bge0 to an identical twin for carp failover. Just simple
routing, no bridging...
Here's the dmesg...
OpenBSD 4.6-stable (GENERIC) #2: Fri Jan 29 16:35:10 CST 2010
r...@dodo.spa.umn.edu:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel(R) Xeon(TM) CPU 3.06GHz ("GenuineIntel" 686-class) 3.06 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID
,xTPR
real mem = 4227264512 (4031MB)
avail mem = 4111294464 (3920MB)
mainbus0 at root
bios0 at mainbus0: AT/286+ BIOS, date 09/29/06, BIOS32 rev. 0 @ 0xffe90,
SMBIOS rev. 2.3 @ 0xfb320 (56 entries)
bios0: vendor Dell Computer Corporation version "A12" date 09/29/2006
bios0: Dell Computer Corporation PowerEdge 1750
acpi0 at bios0: rev 0
acpi0: tables DSDT FACP APIC SPCR
acpi0: wakeup devices RTC_(S5) PCI0(S5) PCI3(S5) PCI2(S5) PCI1(S5)
acpitimer0 at acpi0: 3579545 Hz, 32 bits
acpimadt0 at acpi0 addr 0xfee00000: PC-AT compat
cpu0 at mainbus0: apid 0 (boot processor)
cpu0: apic clock running at 132MHz
cpu at mainbus0: not configured
ioapic0 at mainbus0: apid 8 pa 0xfec00000, version 11, 16 pins
ioapic0: misconfigured as apic 0, remapped to apid 8
ioapic1 at mainbus0: apid 9 pa 0xfec01000, version 11, 16 pins
ioapic1: misconfigured as apic 0, remapped to apid 9
ioapic2 at mainbus0: apid 10 pa 0xfec02000, version 11, 16 pins
ioapic2: misconfigured as apic 0, remapped to apid 10
acpiprt0 at acpi0: bus 0 (PCI0)
acpiprt0: no apic found for irq -1
acpiprt1 at acpi0: bus 4 (PCI4)
acpiprt2 at acpi0: bus 3 (PCI3)
acpiprt3 at acpi0: bus 2 (PCI2)
acpiprt4 at acpi0: bus 1 (PCI1)
acpicpu0 at acpi0
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x1000 0xc9000/0x4000
0xcd000/0x1800 0xce800/0x600 0xec000/0x4000!
ipmi at mainbus0 not configured
pci0 at mainbus0 bus 0: configuration mode 1 (bios)
pchb0 at pci0 dev 0 function 0 "ServerWorks CNB20-HE Host (GC-LE)" rev 0x33
pchb1 at pci0 dev 0 function 1 "ServerWorks CNB20-HE Host (GC-LE)" rev 0x00
pci1 at pchb1 bus 1
em0 at pci1 dev 4 function 0 "Intel PRO/1000MT (82546GB)" rev 0x03: apic
9 int 4 (irq 7), address 00:04:23:c5:24:0e
em1 at pci1 dev 4 function 1 "Intel PRO/1000MT (82546GB)" rev 0x03: apic
9 int 5 (irq 6), address 00:04:23:c5:24:0f
pchb2 at pci0 dev 0 function 2 "ServerWorks CNB20-HE Host (GC-LE)" rev 0x00
pci2 at pchb2 bus 3
"Dell DRAC 4 Embedded/Optional" rev 0x00 at pci0 dev 8 function 0 not
configured
puc0 at pci0 dev 8 function 1 "Dell DRAC 3 Virtual UART" rev 0x00:
ports: 1 com
com3 at puc0 port 0 apic 9 int 13 (irq 14): ns16550a, 16 byte fifo
com3: probed fifo depth: 0 bytes
"Dell DRAC 3 Embedded/Optional" rev 0x00 at pci0 dev 8 function 2 not
configured
vga1 at pci0 dev 14 function 0 "ATI Rage XL" rev 0x27
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
piixpm0 at pci0 dev 15 function 0 "ServerWorks CSB5" rev 0x93: SMBus
disabled
pcib0 at pci0 dev 15 function 3 "ServerWorks CSB5 LPC" rev 0x00
pchb3 at pci0 dev 16 function 0 "ServerWorks CIOB-E" rev 0x12
pchb4 at pci0 dev 16 function 2 "ServerWorks CIOB-E" rev 0x12
pci3 at pchb4 bus 2
bge0 at pci3 dev 0 function 0 "Broadcom BCM5704C" rev 0x02, BCM5704 A2
(0x2002): apic 9 int 0 (irq 5), address 00:0f:1f:66:a4:93
brgphy0 at bge0 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
bge1 at pci3 dev 0 function 1 "Broadcom BCM5704C" rev 0x02, BCM5704 A2
(0x2002): apic 9 int 1 (irq 4), address 00:0f:1f:66:a4:94
brgphy1 at bge1 phy 1: BCM5704 10/100/1000baseT PHY, rev. 0
pchb5 at pci0 dev 17 function 0 "ServerWorks CIOB-X2 PCIX" rev 0x05
pchb6 at pci0 dev 17 function 2 "ServerWorks CIOB-X2 PCIX" rev 0x05
pci4 at pchb6 bus 4
mpi0 at pci4 dev 5 function 0 "Symbios Logic 53c1030" rev 0x07: apic 9
int 2 (irq 15)
scsibus0 at mpi0: 16 targets, initiator 7
sd0 at scsibus0 targ 0 lun 0: <IBM, IC35L036UCDY10-0, S27T> SCSI3
0/direct fixed
sd0: 34732MB, 512 bytes/sec, 71132959 sec total
sd1 at scsibus0 targ 1 lun 0: <IBM, IC35L036UCDY10-0, S27T> SCSI3
0/direct fixed
sd1: 34732MB, 512 bytes/sec, 71132959 sec total
safte0 at scsibus0 targ 6 lun 0: <PE/PV, 1x3 SCSI BP, 1.1> SCSI2
3/processor fixed
mpi0: target 0 Sync at 160MHz width 16bit offset 127 QAS 0 DT 1 IU 1
mpi0: target 1 Sync at 160MHz width 16bit offset 127 QAS 0 DT 1 IU 1
mpi1 at pci4 dev 5 function 1 "Symbios Logic 53c1030" rev 0x07: apic 9
int 3 (irq 14)
scsibus1 at mpi1: 16 targets, initiator 7
isa0 at pcib0
isadma0 at isa0
com1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pmsi0 at pckbc0 (aux slot)
pckbc0: using irq 12 for aux slot
wsmouse0 at pmsi0 mux 0
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
npx0 at isa0 port 0xf0/16: reported by CPUID; using exception 16
mtrr: Pentium Pro MTRR support
softraid0 at root
root on sd0a swap on sd0b dump on sd0b
