On Tue, 2006-03-07 at 13:49 -0800, Stephen Hemminger wrote:
> On Tue, 07 Mar 2006 13:44:51 -0800
> Matt Leininger <[EMAIL PROTECTED]> wrote:
>
> > On Mon, 2006-03-06 at 19:13 -0800, Shirley Ma wrote:
> > >
> > > > More likely you are getting hit by the fact that TSO prevents the
> > > congestion
> > > window from increasing properly. This was fixed in 2.6.15 (around mid
> > > of Nov 2005).
> > >
> > > Yep, I noticed the same problem. After updating to the new kernel, the
> > > performance are much better, but it's still lower than before.
> >
> > Here is an updated version of OpenIB IPoIB performance for various
> > kernels with and without one of the TSO patches. The netperf
> > performance for the latest kernels has not improved the TSO performance
> > drop.
> >
> > Any comments or suggestions would be appreciated.
> >
> > - Matt
>
> Configuration information? like did you increase the tcp_rmem, tcp_wmem?
> Tcpdump traces of what is being sent and available window?
> Is IB using NAPI or just doing netif_rx()?
I used the standard setting for tcp_rmem and tcp_wmem. Here are a
few other runs that change those variables. I was able to improve
performance by ~30MB/s to 403 MB/s, but this is still a ways from the
474 MB/s before the TSO patches.
Thanks,
- Matt
All benchmarks are with RHEL4 x86_64 with HCA FW v4.7.0
dual EM64T 3.2 GHz PCIe IB HCA (memfull)
patch 1 - remove changeset 314324121f9b94b2ca657a494cf2b9cb0e4a28cc
msi_x=1 for all tests
Kernel OpenIB netperf (MB/s)
2.6.16-rc5 in-kernel 403
tcp_wmem 4096 87380 16777216 tcp_rmem 4096 87380 16777216
2.6.16-rc5 in-kernel 395
tcp_wmem 4096 102400 16777216 tcp_rmem 4096 102400 16777216
2.6.16-rc5 in-kernel 392
tcp_wmem 4096 65536 16777216 tcp_rmem 4096 87380 16777216
2.6.16-rc5 in-kernel 394
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 102400 16777216
2.6.16-rc5 in-kernel 377
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 153600 16777216
2.6.16-rc5 in-kernel 377
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 131072 16777216
2.6.16-rc5 in-kernel 353
tcp_wmem 4096 262144 16777216 tcp_rmem 4096 262144 16777216
2.6.16-rc5 in-kernel 305
tcp_wmem 4096 262144 16777216 tcp_rmem 4096 524288 16777216
2.6.16-rc5 in-kernel 303
tcp_wmem 4096 131072 16777216 tcp_rmem 4096 524288 16777216
2.6.16-rc5 in-kernel 290
tcp_wmem 4096 524288 16777216 tcp_rmem 4096 524288 16777216
2.6.16-rc5 in-kernel 367 default tcp values
--------------------
All with standard tcp settings
Kernel OpenIB netperf (MB/s)
2.6.16-rc5 in-kernel 367
2.6.15 in-kernel 382
2.6.14-rc4 patch 12 in-kernel 436
2.6.14-rc4 patch 1 in-kernel 434
2.6.14-rc4 in-kernel 385
2.6.14-rc3 in-kernel 374
2.6.13.2 svn3627 386
2.6.13.2 patch 1 svn3627 446
2.6.13.2 in-kernel 394
2.6.13-rc3 patch 12 in-kernel 442
2.6.13-rc3 patch 1 in-kernel 450
2.6.13-rc3 in-kernel 395
2.6.12.5-lustre in-kernel 399
2.6.12.5 patch 1 in-kernel 464
2.6.12.5 in-kernel 402
2.6.12 in-kernel 406
2.6.12-rc6 patch 1 in-kernel 470
2.6.12-rc6 in-kernel 407
2.6.12-rc5 in-kernel 405
2.6.12-rc5 patch 1 in-kernel 474
2.6.12-rc4 in-kernel 470
2.6.12-rc3 in-kernel 466
2.6.12-rc2 in-kernel 469
2.6.12-rc1 in-kernel 466
2.6.11 in-kernel 464
2.6.11 svn3687 464
2.6.9-11.ELsmp svn3513 425 (Woody's results, 3.6Ghz EM64T)
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
