Greg 'groggy' Lehey wrote:
Last week, at the Linux.conf.au in Dunedin, Van Jacobson presented
some slides about work he has been doing rearchitecting the Linux
network stack. He claims to have reduced the CPU usage by 80% and
doubled network throughput (he expects more, but it was limited by
memory bandwidth). The approach looks like it would work on FreeBSD
as well. I spoke to him and he confirmed.
He's currently trying to get the code released as open source, but in
the meantime his slides are up on
http://www.lemis.com/grog/Documentation/vj/. Yes, this is my web
site. The conference organizers are going to put it up on their web
site soon, but in the meantime he's asked me to put it were I can.
Comments?
It's an interesting approach. However there are a few caveats which
put its probable overall performance on par or again with the traditional
sockets approach.
In his model the buffer (window) resides within user space and is shared
with the kernel. This is very losely related to our zero-copy page flipping
socket buffer. However this doesn't solve the problem of socket buffer
memory overcommit. In fact with his model memory actually in use at any
given point in time may be a lot more than the always fully committed socket
buffer (in userland share with kernel) plus a number of outstanding packets
waiting in the socket queue. The shared user/kernel socket buffer should
not be paged out and thus must stay resident. With a large numbers of
connections on a machine this gets inefficient because all buffer memory
is always committed and not just when it is needed. Memory overcommit
goes away.
Processing the TCP segments on the same CPU as the userland resides (provided
it doesn't migrate [too often]) is certainly beneficial and something we are
looking at for some time already. However we are not there yet and have still
some work on the TCP stack to do for this to become a reality.
Processing the TCP segments within the process CPU quantum and only when it
gets selected by the scheduler is a very interesting idea. It has a couple
of true advantages and theoretical disadvantages. On the good side it accounts
the work in the TCP stack to the process, aggregates processing all segments
that arrived between process runs and keeps good cpu/cache locality. On the
potential negative side it increases segment latency and has to maintain not
only the socket buffer but also another unprocessed-packet buffer. The packet
buffer has to be limited or we open ourselfs up to memory exhaustion attacks.
When many packets for a connection arrive and the process doesn't get scheduled
quickly enough we may get packet loss because packet queue overflows. This can
be dealt with in relatively good ways though.
Summary: Some gems in there and we are certainly looking at a couple of those
ideas to adapt to our network stack in the future.
--
Andre
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
