Date: Fri, 29 Jan 2016 12:44:07 +0900 From: Ryota Ozaki <ozak...@netbsd.org>
So here is a complete patch: http://www.netbsd.org/~ozaki-r/softint-if_input-full.diff With the patch if_input and above should not run in HW interrupt anymore. Instead of adding fields to the already overpopulated struct ifnet, how about creating a new struct if_percpuq or something which drivers can voluntarily use if they desire? struct if_percpuq { void *ipq_si; struct percpu *ipq_ifqs; /* struct ifqueue */ }; struct if_percpuq * if_percpuq_create(void (*)(void *), void *); void if_percpuq_destroy(struct if_percpuq *); void if_percpuq_enqueue(struct if_percpuq *, struct mbuf *); To check at compile-time to make sure all callers have been converted to the softint-only ifp->if_input, we could change its signature to add a dummy argument, or rename the member. The signature/name of ifp->if_input is also closer to the real change -- forbidding *_input in hardintr context -- than the signature of if_initialize. The reason I suggest a separate API which drivers can voluntarily use is that I expect we may want alternatives -- e.g., drivers without multiqueue support may want to use a pktq instead and automatically distribute to other CPUs[*]. I've also never liked initialization flags like IF_INPUTF_NO_SOFTINT to suppress a newly-default option. Also: I think your patch is broken as is for all USB NICs -- all their `interrupt' routines actually run in softint context, if I'm not mistaken, so this kassert will fire: +void +if_input(struct ifnet *ifp, struct mbuf *m) +{ + + if (ifp->if_input_ifqs) { + struct ifqueue *ifq = percpu_getref(ifp->if_input_ifqs); + + KASSERT(cpu_intr_p()); The same may go for some other NICs, such as se(4), for which there is a possible use from thread context which I can't rule out in thirty seconds. Another possible case is xennet(4). [*] Currently this happens (or could happen) with ip_pktq anyway, but if I recall correctly, on discussion, rmind@ agreed with me that that's the wrong place and because of variation in hardware support for rx multiqueue or hashing flows, it should be the NIC driver's job to decide how to distribute packets -- even if that means explicitly calling a software abstraction to make the decision when the NIC can't do rx multiqueue or hash flows itself.
