Hi, have you tried timeout and optimization settings in PF ? try : set optimization high-latency or set optimization conservative
also try set timeout ..... and watch the limits of state tables and set the properly if needed. Regards On Sat, Jan 7, 2012 at 5:17 AM, Graham Allan <al...@physics.umn.edu> wrote: > Hi, > > I'm trying to troubleshoot some performance issues for high speed data > transfers across a long network path with a fairly high bandwidth delay. > > The router is running OpenBSD 4.6 - admittedly well overdue for an update > to 5.0, but I did briefly test 5.0 on a backup machine and saw much the > same. The hardware is a Dell 1750 (getting old but seemed to be keeping up) > with Intel PRO/1000MT interfaces (82546GB) inside and out. It's been in use > for a number of years, passing 750-800Mbps traffic quite happily. > > sysctl tuning is limited to raising net.inet.ip.maxlen from 256 -> 1024. > > We now have a need for high bandwidth data transfers to a Physics lab > across the world, and with some help from our network group have a route > set up which can pass 700Mbps - but only when our local endpoint is outside > the OpenBSD router. > > Doing local tests with iperf using linux boxes as the endpoints we can > pass up to 750Mbps with ~0.3% UDP packet loss, whether that traffic goes > through the router or not. The OpenBSD machine itself shows no packet drops > with "netstat -id" or "sysctl net.inet.ip.drops", and no Ierrs on the > network interfaces. > > However long-distance iperf tests to the distant lab show very different > results... > > through router: 10/80 Mbps (in/out) > bypassing router: 400/750 Mbps (in/out) > > so clearly the router is affecting *something*. All the data suggests it's > not dropping packets. Could it be interfering with window scaling? > There is a pf ruleset in place, of course, but since stateful tracking > became the default years ago (at which time I removed most of the "keep > state" bits), it should be hard to misconfigure that...? There's nothing > very exotic in pf.conf - default deny and a bunch of "pass quick" rules for > various services; no NAT-type rules. Tried disabling packet scrubbing on a > whim, but no change. > > One interesting point is that for the 750Mbps local iperf tests through > the router I do see fairly high interrupt load (~50%), which I'd expect... > but for the very poor long-distance iperf tests I see very low load. > > Both the linux endpoints have been tuned for the high BDP (the network > latency is around 250ms - so huge tcp buffers to cover the in-flight data), > but none of these values are relevant for packet forwarding on the router... > > I'm lost for where to look next, and would appreciate any ideas! > > Graham > --
