Date: Fri, 15 Jan 2016 18:35:32 +0900 From: Ryota Ozaki <ozak...@netbsd.org>
BTW, I forgot noting that ipi_sysinit in main() needed a tweak to use pktq_create in driver attach routines. The tweak is like this finally: http://www.netbsd.org/~ozaki-r/fix-ipi_sysinit.diff I'm a little concerned about the number of IPIs you may introduce here, if you create one pktq per interface, which led me to a concern about the efficacy of distributing packets across CPUs twice. Softints and IPIs are both scarce resources, but I think IPIs are much more scarce: as currently set up, the kernel can have ~hundreds of softints, but only 32 IPIs. As far as I can tell, the only reason pktq_create establishes an IPI is to allow executing softints on remote CPUs, so that we can distribute the load of processing IP packets across multiple CPUs. (Is this an advantage over just using device multiqueue support to distribute IP packet processing? I don't know -- rmind?) If ether_input is going to distribute packets to other CPUs at some point anyway, it seems to me that it's a waste of time to do that twice -- aside from the issue of IPI scarcity. So perhaps we should have either (a) a per-CPU pktq that never distributes packets to another CPU, or (b) a single-CPU pktq, to be used only from the CPU to which the device's (queue's) interrupt handler is bound. I suppose (a) is just a softint together with a percpu(9) of ifqs, and (b) is just a softint together with an ifq.
