I'm picking up the Gigabit Ethernet conversation from a few weeks ago. Harald Welte said: > I'm actually not referring to the Ethernet hardware side, but to the > software stack. Unless you want to write a special 'ursp network stack' > that sits directly on top of the hardware driver (like PF_RING, but > that's read only), you will go through the whole 'normal' Linux network > stack. > > For every packet you process, there's an enormous amount of overhead > you're going through, even if you choose to not use IP and play some > tricks with raw sockets. Due to the sockets-based interface, you will > have additional copying of the data. > > I'm by no means arguing that the linux network stack is slow. It's just > not intended for an application which just wants to get big data streams > with low latency and little overhead into userspace.
I'm surprised to hear you suggest this (though I note you didn't give numbers, just generalizations like "enormous"). The first time I looked seriously at a network stack was in a Van Jacobson/Mike Karels class about how they made BSD TCP on Sun-3/50s run at 1 MByte/sec, within a few % of the theoretical maximum thruput of 10Mbit/s Ethernet. The last time I looked at the Linux network stack was when Dave Miller was cutting his teeth on the SPARC Linux port and making 100MB/s Ethernet run within a few % of theoretical maximum (he quit when he exceeded 10MBytes/sec over TCP from user process to user process). I had of course expected that somebody would do the same thing for Gigabit Ethernet. This appears to have been done; GigE cards seem to be able to push large amounts of data under Linux. What thruput do YOU see using a good GigE card under Linux? Can you move 100 Mbytes/sec from user process to user process with TCP, across two GigE interfaces and a GigE switch? If so, it's doing much better than USB2 ever will, and much better than any version of FireWire. Here's a msg that says Linux was getting 100MBps (presumably MBytes/s) on 2.6.0-test2 in 2003: http://www.uwsg.iu.edu/hypermail/linux/net/0308.1/0037.html Here's one that says 930 Mbps using 2.4.21 on a BiXeon in 2004: http://oss.sgi.com/projects/netdev/archive/2004-04/msg00474.html Here's one that shows 870 Mbits/s in 2002 on 2.4.18 (long haul from Sunnyvale to Chicago): http://www-iepm.slac.stanford.edu/monitoring/bulk/fast/ This msg says that with a 10GB/s Ethernet card (note: 10x as fast as GigE) they can send UDP at 1.9GBps, and send TCP at 2.2Gbps from Chicago to Switzerland. Clearly the Linux kernel overhead isn't keeping thruput below a gigabit: http://www-iepm.slac.stanford.edu/monitoring/bulk/10ge/ There is a lot of research about how to make TCP run at gigabit speeds over *long distances*, particularly in recovering from dropped packets. Nobody seems to need to do research on how to make TCP run at gigabit speeds across the room; it already works. As does UDP. Harald, in what way is your experience different? John _______________________________________________ Discuss-gnuradio mailing list Discuss-gnuradio@gnu.org http://lists.gnu.org/mailman/listinfo/discuss-gnuradio
