On 21 August 2016 at 07:42, Robert N. M. Watson <rwat...@freebsd.org> wrote: > On 20 Aug 2016, at 21:27, Adrian Chadd <adr...@freebsd.org> wrote: > >> I wonder if the right-er thing to do here is to allow the cpuid to be >> whatever it needs to be, but limit the cpuid lookups when it resolves >> to a netisr array. >> >> that'd mean the hybrid model would still return the current CPU up to >> the max CPU id, but anything trying to queue into a netisr would not >> overflow the netisr queue count. > > > I think some refinement of model is required. The original intent was that > “workstreams” represented ordered sets of queues being delivered for deferred > dispatch across the set of CPUs (hence “nws” being allocated using DPCPU). > When performing direct dispatch, as a matter of convenience, we also bill > work performed on a CPU to its workstream: > > 1109 /* > 1110 * If direct dispatch is forced, then unconditionally dispatch > 1111 * without a formal CPU selection. Borrow the current CPU's > stats, > 1112 * even if there's no worker on it. In this case we don't > update > 1113 * nws_flags because all netisr processing will be source > ordered due > 1114 * to always being forced to directly dispatch. > 1115 */ > > In hybrid mode, the world is a little different, because we are willing to > direct dispatch hybrid work *only* if the worker thread isn’t already > processing the workstream: > > 1147 /*- > 1148 * We are willing to direct dispatch only if three conditions > hold: > 1149 * > 1150 * (1) The netisr worker isn't already running, > 1151 * (2) Another thread isn't already directly dispatching, and > 1152 * (3) The netisr hasn't already been woken up. > 1153 */ > > This is important because if the workstream already has a backlog of work, > but the thread isn’t yet running, we must enqueue the work to ensure it > remains ordered with respect to other work in the workstream. > > Assuming we wish to retain the current workstream model, then we need to > adapt the code a bit to handle the case where we still have one workstream > per CPU, but where we are only willing to perform deferred dispatch (or > hybrid dispatch) to a CPU that has a worker thread. We would then bill to any > CPU’s workstream for true direct dispatch, but for hybrid dispatch, continue > to always bill a CPU that has a worker thread to which we were willing to > assign the work. > > I.e., as I think you’re suggesting, we probably need to tweak the functions > slightly to differentiate between “you can run it here and must use curcpu’s > workstream” and “you can run it here but must use a specific CPU’s > workstream” — the former being true direct dispatch, and the latter being > hybrid dispatch where the netisr in question isn’t already running, but we > bill it to that workstream/netisr even though we run it on the current CPU. > > Does this make sense?
Right. Let me go and look into it a little more. I think we may want to revert the change (which just landed to -11, so maybe revert that too) so I can test both of them out for correctness. Andrey, I'm sorry for suggesting we back it out, but I'd like to make sure we don't break networking on 11. :) Is that okay? I will look at this tonight/tomorrow. -adrian _______________________________________________ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
