On 08.11.2012 03:38, YongHyeon PYUN wrote:
On Wed, Nov 07, 2012 at 06:15:30PM -0800, Adrian Chadd wrote:
If so, may I suggest we perhaps accelerate discussing if_transmit() of
multiple frames per call?
Hmm, actually I'm still not a fan of if_transmit() at this moment.
Honestly I don't have good queuing code in driver to handle queue
full condition. Interactions with altq(9) is also one of my
concern as well as packet reordering issue of drbr(9) interface.
The whole interface packet handoff needs some serious reconsideration.
These days we have two queues/buffers at the interfaces. One is the
DMA ring which can take a considerable number of packets and the second
one is the ifq (if enabled). The DMA ring already adds significant
depth and latency so that a packet scheduler like altq(9) become almost
useless. Also modern queue management algorithms like CoDel don't work
with the current framework. Also bufferbloat is a major concern. See
ACM queue article by Jim Gettys.
What we need to take make this functionality available again is a well
specified and reasonably simple interface handoff. It should include
information on the maximum tx DMA ring depth and the current depth.
There should also be a function to limit the current depth to a certain
value.
What I'd like to see is this (names are not fixed):
if_send() as the main entry point for the stack. It's a function
pointer within struct ifnet. In normal operation it is the same
as if_transmit() and directly adds a packet to the tx DMA ring.
Locking of the DMA ring is done in this function and a property
of the driver. The stack always calls unlocked. Obviously the
tx DMA ring lock must not be a sleep lock.
When altq(9) or equivalent is active this function pointer is
replaced with a call to the alternative queuing function that
does it's magic. Again locking of the queuing mechanism is the
property of that mechanism. When a NIC has multiple queues that
it can bind to CPU's locking may not be necessary. We gain this
flexibility in the driver to do that.
if_transmit() is a function pointer for a function that directly
adds a packet to the tx DMA ring (if a free slot is available).
It is never called by the stack directly except in special
circumstances. The altq(9), if active, uses if_transmit() to
add packets to the tx DMA ring. If not active, if_send() is
this function pointer.
if_txeof() is a function pointer for a callback from the driver
to an altq(9) dequeue function, if active. It is called when
when new free slots on the tx DMA ring are available.
When a driver needs a software interface queue because the tx DMA
is too small, then the stack should provide a generic queuing
implementation the driver can use.
I've begun to explore this area while hacking on bge(4) and em(4)
in tcp_workqueue branch.
It's a very interesting path and we going to have a couple more
discussions before we arrive at the optimal solution. :)
--
Andre
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
