On Fri, 14 Oct 2005, Michael VInce wrote:
I been doing some network benchmarking using netperf and just simple
'fetch' on a new network setup to make sure I am getting the most out of
the router and servers, I thought I would post some results in case some
one can help me with my problems or if others are just interested to see
the results.
Until recently (or maybe still), netperf was compiled with -DHISTOGRAM by
our port/package, which resulted in a significant performance drop. I
believe that the port maintainer and others have agreed to change it, but
I'm not sure if it's been committed yet, or which packages have been
rebuilt. You may want to manually rebuild it to make sure -DHISTOGRAM
isn't set.
You may want to try setting net.isr.direct=1 and see what performance
impact that has for you.
Robert N M Watson
The network is currently like this, where machines A and B are the Dell 1850s
and C is the 2850 x 2 CPU (Server C has Apache2 worker MPM on it) and server
B is the gateway and A is acting as a client for fetch and netperf tests.
A --- B --- C
The 2 1850s are running AMD64 Freebsd 6.0rc1 (A and B) while C is running
5.4-stable i386 from Oct 12
My main problem is that if I compile SMP into the machine C (5.4stable) the
network speed goes down to a range between 6mbytes/sec to 15mbytes/sec on
SMP.
If I use GENERIC kernel the performance goes up to what I have show below
which is around 65megabytes/sec for a 'fetch' get test from Apache server and
933mbits/sec for netperf.
Does any know why why network performance would be so bad on SMP?
Does any one think that if I upgrade the i386 SMP server to 6.0RC1 the SMP
network performance would improve? This server will be running java so I need
it to be stable and is the the reason I am using i386 and Java 1.4
I am happy with performance of direct machine to machine (non SMP) which is
pretty much full 1gigabit/sec speeds.
Going through the gateway server-B seems to drop its speed down a bit for in
and out direction tcp speed tests using netperf I get around 266mbits/sec
from server A through gateway Server-B to server-C which is quite adequate
for the link I currently have for it.
Doing a 'fetch' get for a 1gig file from the Apache server gives good speeds
of close to 600mbits/sec but netperf shows its weakness with 266mbits/sec.
This is as fast as I need it to be but does any one know the weak points on
the router gateway to make it faster? Is this the performance I should expect
for FreeBSD as a router with gigabit ethers?
I have seen 'net.inet.ip.fastforwarding' in some peoples router setups on the
list but nothing about what it does or what it can affect.
I haven't done any testing with polling yet but if I can get over
900mbits/sec on the interfaces does polling help with passing packets from
one interface to the other?
All machines have PF running other then that they don't really have any
sysctls or special kernel options.
Here are some speed benchmarks using netperf and 'fetch' gets.
Server A to server C with server C using SMP kernel and just GENERIC kernel
further below
B# /usr/local/netperf/netperf -l 10 -H server-C -t TCP_STREAM -i 10,2 -I 99,5
-- -m 4096 -s 57344 -S 57344
TCP STREAM TEST to server-C : +/-2.5% @ 99% conf. : histogram
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
57344 57344 4096 10.06 155.99
tank# fetch -o - > /dev/null http://server-C/file1gig.iso
- 100% of 1055 MB 13 MBps
00m00s
##### Using generic non SMP kernel
Server A to server C with server C using GENERIC kernel.
A# fetch -o - > /dev/null http://server-C/file1gig.iso
- 100% of 1055 MB 59 MBps
00m00s
A# ./tcp_stream_script server-C
/usr/local/netperf/netperf -l 60 -H server-C -t TCP_STREAM -i 10,2 -I 99,5 --
-m 4096 -s 57344 -S 57344
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
57344 57344 4096 60.43 266.92
------------------------------------
###############################################
Connecting from server-A to B (gateway)
A# ./tcp_stream_script server-B
------------------------------------
/usr/local/netperf/netperf -l 60 -H server-B -t TCP_STREAM -i 10,2 -I 99,5 --
-m 4096 -s 57344 -S 57344
TCP STREAM TEST to server-B : +/-2.5% @ 99% conf. : histogram
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
57344 57344 4096 61.80 926.82
------------------------------------
##########################################
Connecting from server B (gateway) to server C
Fetch and Apache2 test
B# fetch -o - > /dev/null http://server-C/file1gig.iso
- 100% of 1055 MB 74 MBps
00m00s
Netperf test
B# /usr/local/netperf/tcp_stream_script server-C
/usr/local/netperf/netperf -l 60 -H server-C -t TCP_STREAM -i 10,2 -I 99,5 --
-m 4096 -s 57344 -S 57344
TCP STREAM TEST to server-C : +/-2.5% @ 99% conf. : histogram
Recv Send Send
Socket Socket Message Elapsed
Size Size Size Time Throughput
bytes bytes bytes secs. 10^6bits/sec
57344 57344 4096 62.20 933.94
------------------------------------
Cheers,
Mike
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
