This is a well known issues called "Long Fat Pipe Network". There's many university papers about it and many tricks to get around it on software-based boxes.
Adjusting your TCP window size was the best start, if it's set properlu. The basic formula is provided in this forum post : http://forums.whirlpool.net.au/archive/116165 Good luck ! 2011/6/28 Eugen Leitl <eu...@leitl.org>: > ----- Forwarded message from William Salt <williamejs...@googlemail.com> ----- > > From: William Salt <williamejs...@googlemail.com> > Date: Tue, 28 Jun 2011 08:03:25 +0100 > To: supp...@pfsense.com > Subject: [pfSense Support] Strange TCP connection behavior 2.0 RC2 (+3) > Reply-To: supp...@pfsense.com > > Hi All, > For the last couple of months i have been pulling my hair out > trying to solve this problem. > We have a 1Gbps transatlantic link from the UK to the US, which has > successfully passed the RFC2544 test. > > At either end, we have a media converter, and a supermicro server with an > intel quad port NIC running pfsense 2 (RC2 at one end RC3 at the other) and > the IGB driver on the quad port. > > We can pass 1gbps either way with UDP. However we are experiencing very > strange issues with tcp connections. > > With window scaling enabled, and a max socket buffer set to 16MB, we see no > difference. > Even disabling window scaling and setting the window to 16MB makes no > difference. > > Each TCP connection starts very slowly, and will max out at around 190mbps, > taking nearly 2 minutes to climb to this speed before *plateauing*. > > We have to initiate many (5+) connections to saturate the link with tcp > connections with iperf. > > Real world tests transferring files, max out at 100mbps, using multiple > connections. > > I have followed guides like this: > http://www.psc.edu/networking/projects/tcptune/#FreeBSD > > With no luck, and have tweaked, disabled, and enabled nearly every relevant > sysctl parameter with no luck. > > Can anyone shed some light on this? > > I am now doubting the IGB driver, and am looking to swap out the cards as a > last ditch effort. > However, we have tried different hardware (L3 switches, media convertes + > laptops etc), and the symptoms still persist... > The only constant is freebsd 8.1 - pfsense (or 8.2 for our production > systems). > I have tried the freebsd net mailinglist, but im hoping you lot can help me! > > Cheers in advance > Will > > ----- End forwarded message ----- > -- > Eugen* Leitl <a href="http://leitl.org";>leitl</a> http://leitl.org > ______________________________________________________________ > ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org > 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE > > -- Jérôme Nicolle 06 19 31 27 14
