On 12-11-12 03:27 PM, Andre Oppermann wrote:
On 12.11.2012 20:15, Karim wrote:
Hi all,
I have been following the current discussions on igb tx/rx locking
with great interest and I though
I would point out (as it was before pointed out in kern/138392) that
any driver setting if_start to
NULL and using if_transmit (multi-queue or not) will effectively
break ALTQ.
The reason for this can be found in tbr_timeout() in altq_subr.c
where the token bucket regulator
goes through all ALTQ registered interface and tries to 'kick' them
into sending through if_start().
tbr_timeout() :
...
for (ifp = TAILQ_FIRST(&V_ifnet); ifp;
ifp = TAILQ_NEXT(ifp, if_list)) {
/* read from if_snd unlocked */
if (!TBR_IS_ENABLED(&ifp->if_snd))
continue;
active++;
if (!IFQ_IS_EMPTY(&ifp->if_snd) &&
ifp->if_start != NULL) /*
if_start is NULL if if_transmit is
used in em/igb driver */
(*ifp->if_start)(ifp);
}
...
As you can see if_start is NULL on those new multi-queue enabled
drivers which has for net effect to
'break' ALTQ's token bucket regulator.
I am writing this because I am interested in your comments on how
this can be fixed properly looking
forward. The whole range of suggestions; from 'don't compile with
EM_MULTIQUEUE defined' to 'here is
how you can make ALTQ use drbr' will help.
This whole area needs some serious re-consideration for 10.0.
Even without the if_start issue ALTQ is somewhat broken because
the DMA rings nowadays are just too deep.
See this recent message to this list for some thoughts:
http://lists.freebsd.org/pipermail/freebsd-net/2012-November/033780.html
On top of refining the stack/driver boundary I'm thinking of a
dedicated ethernet layer to consolidate all the ethernet extensions,
have a single place to go and to prevent the drivers from re-inventing
the wheel all over every time.
I'm working towards it and leaning into various drivers while trying
to bring in the hybrid interrupt/polling mode with life-lock prevention.
It'll take a few weeks until I'm ready to show a stack/ethernet/driver
prototype for discussion. Parts may surface earlier in my tcp_workqueue
svn branch.
Hi,
Glad to see someone looking into this :)
The addition of a common layer for queuing algorithm is an interesting
idea and makes me wonder how alternate queuing techniques would be able
to use this shim layer to leverage devices with multiple queues.
The current limitation (in terms of performances) with ALTQ going
forward is its current inability to use multiple queues, in part because
it was left out when the current (drbr) implementation of multiqueues
was made but mainly because of its current internal structure (using a
single global IFQ lock).
Is it part of the plan to abstract multiqueuing from alternate queuing
algorithm or will it still be left to ALTQ, through driver managed
queues for example, to manage the TX DMA ring lock?
Thanks,
Karim.
_______________________________________________
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
