On 07.08.2013 09:18, Luigi Rizzo wrote:
On Wed, Aug 7, 2013 at 5:26 AM, Mike Karels <m...@karels.net <mailto:m...@karels.net>> wrote: Jumping to (near) the end of the thread, I like most of Andre's proposal. Running with minimal locks at this layer is an admirable goal, and I agree with most of what was said. I have a few observations on the general changes, or related issues:There was mention of taskqueues. I think that with MSI-X, taskqueues should not be needed or used. More specifically, having separate ithreads and taskqueues, with ithreads deferring to taskqueues after some limit, makes sense only for MSI and legacy interrupts. With MSI-X, an interrupt thread should be able to process packets indefinitely with sufficient CPU resources, and there is no reason to context switch to a different thread periodically. A periodic "yield" might be reasonable, but if it is necessary, small packet performance will suffer. However, most of this is internal to the driver. i am not completely clear on what is the difference between ithreads and taskqueues.
The difference between ithreads and taskqueues is actually very small and mostly in name and how they're set up and kicked off when work is to be done. Both are real kernel threads. Most of the confusion, also in Mikes response, seems to come from the name ithreads for interrupt-threads. However an ithread is not running in interrupt context, only the interrupt handler (called filter) does. Scheduling a taskqueue from an ithread is a bit round-about but commonly done. The bus_setup_intr(9) man page isn't helping to clear up the confusion. The idea is that a (legacy) interrupt is handled in two stages: 1) a small function reading the hardware interrupt status register to determine if this hardware actually raised an interrupt, and acknowledge it (to prevent additional interrupts from firing while this one is handled). If it wasn't this card, it hands off to the handler if this interrupt line is shared; 2) the actual function/thread handling the data that was indicated by the interrupt. Only step 1 runs in interrupt context and thus must be very small and avoid any form of blocking. Step 2 is a normal kernel thread set up as an ithread running at an elevated priority compared to user-space processes/threads. MSI and MSI-X always are exclusive and non-shared interrupts. Thus a handler running in interrupt context isn't necessary for non-legacy interrupt sources. The ithread can be scheduled right away to do its work.
Also, Andre's proposal requires to force-kill the ithread, but i am unclear on how to do it safely (i.e. without leaving the data structures in some inconsistent state), unless ithread periodically yields the CPU when it is in a safe state. While this is internal to the driver, we should probably provide some template code to avoid that each driver implements its own way to shutdown the ithread.
Yes, when done a well-tested, stable and performant template will be provided. -- Andre _______________________________________________ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
