On Wed, 17 Nov 2010, Maxim Sobolev wrote:
On 11/16/2010 8:12 AM, Bruce Evans wrote:
This was quite low for yestdeay's uses (starting in about 1995), but today
it is little missed since only yesterday's low-end hardware uses it. Most
of today's interfaces are 1Gbps, and for this it is almost essential for
the hardware to have a ring buffer with > 50 entries, so most of today's
drivers ignore ifqmaxlen and set the queue length to the almost equally
bogus value of the ring buffer size (-1). I set it to about 10000 instead
in bge and em (10000 is too large, but fixes streaming under certain loads
when hz is small).
One of those interfaces is if_rl, which is still quite popular these days and
supports speeds up to 1gbps (which I believe triggered this change). But in
It is the only one on the list that I used. Maybe it should be handled
specially. Just bump up its queue lengths to maybe 128 for 100 Mbps and
512 for 1 Gbps in all cases, or tune this depending on the amount of memory?
general I agree, unfortunately FreeBSD network subsystem is tuned for
yesteday's speeds. We are seeing lot of lookups and other issues under high
PPS. I wish somebody could stand and pick up the task of cleaning it up and
re-tuning eventually for 2010. We could probably even sponsor in part such a
work (anyone).
I haven't seen any lockups, but just the maximum pps on fixed hardware
decreasing with every increase in the FreeBSD version number (about 30%
since FreeBSD-5). My hardware CPU and bus are saturated by low-end em
1 Gbps and medium-end bge 1 Gbps, so bloat in the stack translates into
lower pps. I tuned bge a lot to make it fast under the version of
FreeBSD-5 that I usually run, but barely touched upper layers.
Apart from interface tuning for Gbps speeds, another area that needs more
work is splitting up memory pool for the IPC from the memory pool for the
other networking. Today's software is highly distributed and rock-solid IPC
is a must for the FreeBSD being a solid server application platform. That's
OK when under the load we drop some packets, but it's not OK when extreme
network activity can bring down communications between application and
database system within the host itself. And that's exactly what can happen in
FreeBSD.
Does flow control help here? I think it should prevent most dropped packets,
but be actively harmful if it stops the flow when IPC packets are queued
behind non-IPC ones. Large queue lengths are also bad for latency.
Bruce
_______________________________________________
svn-src-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-all
To unsubscribe, send any mail to "svn-src-all-unsubscr...@freebsd.org"
