On 12.11.2012 20:15, Karim wrote:
Hi all,I have been following the current discussions on igb tx/rx locking with great interest and I though I would point out (as it was before pointed out in kern/138392) that any driver setting if_start to NULL and using if_transmit (multi-queue or not) will effectively break ALTQ. The reason for this can be found in tbr_timeout() in altq_subr.c where the token bucket regulator goes through all ALTQ registered interface and tries to 'kick' them into sending through if_start(). tbr_timeout() : ... for (ifp = TAILQ_FIRST(&V_ifnet); ifp; ifp = TAILQ_NEXT(ifp, if_list)) { /* read from if_snd unlocked */ if (!TBR_IS_ENABLED(&ifp->if_snd)) continue; active++; if (!IFQ_IS_EMPTY(&ifp->if_snd) && ifp->if_start != NULL) /* if_start is NULL if if_transmit is used in em/igb driver */ (*ifp->if_start)(ifp); } ... As you can see if_start is NULL on those new multi-queue enabled drivers which has for net effect to 'break' ALTQ's token bucket regulator. I am writing this because I am interested in your comments on how this can be fixed properly looking forward. The whole range of suggestions; from 'don't compile with EM_MULTIQUEUE defined' to 'here is how you can make ALTQ use drbr' will help.
This whole area needs some serious re-consideration for 10.0. Even without the if_start issue ALTQ is somewhat broken because the DMA rings nowadays are just too deep. See this recent message to this list for some thoughts: http://lists.freebsd.org/pipermail/freebsd-net/2012-November/033780.html On top of refining the stack/driver boundary I'm thinking of a dedicated ethernet layer to consolidate all the ethernet extensions, have a single place to go and to prevent the drivers from re-inventing the wheel all over every time. I'm working towards it and leaning into various drivers while trying to bring in the hybrid interrupt/polling mode with life-lock prevention. It'll take a few weeks until I'm ready to show a stack/ethernet/driver prototype for discussion. Parts may surface earlier in my tcp_workqueue svn branch. -- Andre _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
