Hi,
This problem has been bugging me for month now. It started
happening a month after 3.8 got tagged. At least, that's when I
started noticing it. So it might be anything. But I suspect the
OpenBSD side the most since returning to an older Linux release on
the client from a liveCD didn't fix the problem. The OpenBSD
server doesn't have a CD-drive.
OpenBSD server <-> linux client
Both rtl8169 gigabit networkcards
Uploading to the server goes with 11Mbytes/s, the speedlimit of
the ide harddrives, but the downloading goes with erratic
speeds. 1Mbyte/s at best, 100Kbyte/s most of the time, sometimes
no more than 20Kbytes/s
So I started examining and found a lot of fragmentatation. So I
resolved that first by turning down the packetsize to 1024 for
nfs.
I disabled and enabled scrubbing, it didn't make a difference
either
I also tried disabling the pf firewall completely. No difference
either.
And I disabled net.ipv4.tcp_ecn on both machines. Still the same.
I wonder if anyone has the magical solution.
`sudo tcpdump -pnettti re0 not port ssh' output while nfs file transfer:
------------------------------------------------------------------------
Dec 23 02:09:30.155481 0:8:a1:3c:34:7a 0:8:a1:3c:34:79 0800 186:
172.16.11.3.1022 > 172.16.11.1.2049: xid 0xfdc03144 144 read [|nfs] (DF)
Dec 23 02:09:30.155515 0:8:a1:3c:34:7a 0:8:a1:3c:34:79 0800 186:
172.16.11.3.1022 > 172.16.11.1.2049: xid 0xfec03144 144 read [|nfs] (DF)
Dec 23 02:09:30.155545 0:8:a1:3c:34:7a 0:8:a1:3c:34:79 0800 186:
172.16.11.3.1022 > 172.16.11.1.2049: xid 0xffc03144 144 read [|nfs] (DF)
Dec 23 02:09:30.155766 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 1194:
172.16.11.1.2049 > 172.16.11.3.1022: xid 0xfdc03144 reply ok 1152 read
Dec 23 02:09:30.155896 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 1194:
172.16.11.1.2049 > 172.16.11.3.1022: xid 0xfec03144 reply ok 1152 read
BTW : without -p (promiscuous) mode the connection gets killed in a matter of
seconds.
~% ping -s 8000 172.16.11.3
PING 172.16.11.3 (172.16.11.3): 8000 data bytes
8008 bytes from 172.16.11.3: icmp_seq=0 ttl=64 time=0.819 ms
8008 bytes from 172.16.11.3: icmp_seq=1 ttl=64 time=0.745 ms
8008 bytes from 172.16.11.3: icmp_seq=2 ttl=64 time=0.756 ms
8008 bytes from 172.16.11.3: icmp_seq=3 ttl=64 time=0.737 ms
8008 bytes from 172.16.11.3: icmp_seq=5 ttl=64 time=0.770 ms
8008 bytes from 172.16.11.3: icmp_seq=6 ttl=64 time=0.751 ms
8008 bytes from 172.16.11.3: icmp_seq=8 ttl=64 time=0.770 ms
8008 bytes from 172.16.11.3: icmp_seq=9 ttl=64 time=0.746 ms
8008 bytes from 172.16.11.3: icmp_seq=10 ttl=64 time=0.755 ms
8008 bytes from 172.16.11.3: icmp_seq=11 ttl=64 time=0.749 ms
8008 bytes from 172.16.11.3: icmp_seq=12 ttl=64 time=0.769 ms
8008 bytes from 172.16.11.3: icmp_seq=13 ttl=64 time=0.756 ms
There is lots of packetloss, also very high pingtimes were
reported.
`sudo tcpdump -pnettti re0 not port ssh' output while doing `ping -s 8000
172.16.11.3'
Dec 23 02:14:14.629981 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 1514: 172.16.11.1 >
172.16.11.3: icmp: echo request (frag 36405:[EMAIL PROTECTED])
Dec 23 02:14:14.629990 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 1514: 172.16.11.1 >
172.16.11.3: (frag 36405:[EMAIL PROTECTED])
Dec 23 02:14:14.629995 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 1514: 172.16.11.1 >
172.16.11.3: (frag 36405:[EMAIL PROTECTED])
Dec 23 02:14:14.630001 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 1514: 172.16.11.1 >
172.16.11.3: (frag 36405:[EMAIL PROTECTED])
Dec 23 02:14:14.630008 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 1514: 172.16.11.1 >
172.16.11.3: (frag 36405:[EMAIL PROTECTED])
Dec 23 02:14:14.630013 0:8:a1:3c:34:79 0:8:a1:3c:34:7a 0800 642: 172.16.11.1 >
172.16.11.3: (frag 36405:[EMAIL PROTECTED])
Dec 23 02:14:14.630272 0:8:a1:3c:34:7a 0:8:a1:3c:34:79 0800 1514: 172.16.11.3 >
172.16.11.1: icmp: echo reply (frag 64957:[EMAIL PROTECTED])
I also examined irq conflicts. I removed the usb2 hub which
conflicted with the nic on the openbsd machine. It didn't seem to
matter much.
Here is the ifconfig output from the Linux machine, which shows there are no
network errors or conflicts:
eth0 Link encap:Ethernet HWaddr 00:08:A1:3C:34:7A
inet addr:172.16.11.3 Bcast:172.16.11.255 Mask:255.255.255.0
UP BROADCAST NOTRAILERS RUNNING MULTICAST MTU:1500 Metric:1
RX packets:526113 errors:0 dropped:0 overruns:0 frame:0
TX packets:551835 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:406518025 (387.6 Mb) TX bytes:322415079 (307.4 Mb)
Interrupt:10 Base address:0x2f00
And here is the OpenBSD output:
re0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
lladdr 00:08:a1:3c:34:79
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet 172.16.11.1 netmask 0xffffff00 broadcast 172.16.11.255
inet6 fe80::208:a1ff:fe3c:3479%re0 prefixlen 64 scopeid 0x1
And here is the dmesg:
OpenBSD 3.8-current (GENERIC) #319: Fri Dec 16 15:31:29 MST 2005
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: Intel Pentium II ("GenuineIntel" 686-class, 512KB L2 cache) 412 MHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR
real mem = 200908800 (196200K)
avail mem = 176427008 (172292K)
using 2478 buffers containing 10149888 bytes (9912K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(a3) BIOS, date 06/28/00, BIOS32 rev. 0 @ 0xfb380
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
apm0: flags 70102 dobusy 1 doidle 1
pcibios0 at bios0: rev 2.1 @ 0xf0000/0xb808
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfdf00/144 (7 entries)
pcibios0: PCI Exclusive IRQs: 10 11 12
pcibios0: PCI Interrupt Router at 000:07:0 ("Intel 82371SB ISA" rev 0x00)
pcibios0: PCI bus #1 is the last bus
bios0: ROM list: 0xc0000/0x8800
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "Intel 82443BX AGP" rev 0x03
ppb0 at pci0 dev 1 function 0 "Intel 82443BX AGP" rev 0x03
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "Nvidia Riva TNT" rev 0x04
wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
pcib0 at pci0 dev 7 function 0 "Intel 82371AB PIIX4 ISA" rev 0x02
pciide0 at pci0 dev 7 function 1 "Intel 82371AB IDE" rev 0x01: DMA, channel 0
wired to compatibility, channel 1 wired to compatibility
wd0 at pciide0 channel 0 drive 0: <Maxtor 6Y080L0>
wd0: 16-sector PIO, LBA, 78167MB, 160086528 sectors
wd1 at pciide0 channel 0 drive 1: <Maxtor 6Y160P0>
wd1: 16-sector PIO, LBA48, 156334MB, 320173056 sectors
wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2
wd1(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 2
pciide0: channel 1 disabled (no drives)
uhci0 at pci0 dev 7 function 2 "Intel 82371AB USB" rev 0x01: irq 10
usb0 at uhci0: USB revision 1.0
uhub0 at usb0
uhub0: Intel UHCI root hub, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
"Intel 82371AB Power" rev 0x02 at pci0 dev 7 function 3 not configured
re0 at pci0 dev 11 function 0 "Realtek 8169" rev 0x10: irq 11, address
00:08:a1:3c:34:79
rgephy0 at re0 phy 7: RTL8169S/8110S PHY, rev. 0
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
ne1: irq 10 already in use
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
spkr0 at pcppi0
sysbeep0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
lm0 at isa0 port 0x290/8: LM79
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
isapnp0 at isa0 port 0x279: read port 0x203
ne3 at isapnp0 "Plug & Play Ethernet Card, @@@1980, PNP80D6, " port 0x220/32
irq 5: NE2000 (RTL8019), address 00:c0:26:18:00:02
biomask f745 netmask ff65 ttymask ffe7
pctr: 686-class user-level performance counters enabled
mtrr: Pentium Pro MTRR support
dkcsum: wd0 matches BIOS drive 0x80
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302
# Han
