On 16 October 2012 05:38, John Baldwin <j...@freebsd.org> wrote: > I don't follow how this is related to this thread at all (which has more to do > with ixgbe scheduling duplicate work). However, is your issue that the stack > locks (e.g. socket and protocol layer locks) are held across > if_start/if_transmit?
It's a comment on the larger scale architectural problem. Since if_transmit and if_start are called from multiple thread contexts, the current ways drivers implement this are: * support direct dispatch to hardware, but wrap the whole sending process in one enormous lock, to prevent packet reordering issues; or * drop TX and TX completion into a TX taskqueue (or multiple, one per hardware send queue) and push frames into that taskqueue via some queue and then wake said taskqueue up; or * some bastardised version of both. For the intel drivers, the locks are held for a (potentially) very long time. Both igb and ixgb both hold the locks for the entirety of the TX process. It's not protecting something like a queue operation, it's effectively serialising the entirety of the TX and TX completion process. That works ok-ish for ethernet drivers which are "send and ignore", but for wireless drivers where the stack implements a lot more state, it really does quite suck. And since wireless drivers have a top level idea of sequence and encryption (ie, it's not per-TCP stream, it's across multiple sending streams to a given node), I can't model the locking and serialisation on what the TCP/UDP code does. I wish we had a better way of implementing "serialisation without long, long held locks" but short of stuffing everything into a taskqueue and only locking the send queue involved, I can't really think of anything. Adrian _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
