Andrew Gallatin wrote:
Scott Long writes:
>
> Touching the APIC is tricky. First, you have to pay the cost of a
> spinlock. Then you have to may the cost of at least one read and write
> across the FSB. Even though the APIC registers are memory mapped, they
> are still uncached. It's not terribly expensive, but it does add up.
> Bypassing this and using a fast interrupt means that you pay the cost of
> 1 PCI read, which you would have to do anyways with either method, and 1
> PCI write, which will be posted at the host-pci bridge and thus only as
> expensive as an FSB write. Overall, I don't think that the cost
> difference is a whole lot, but when you are talking about thousands of
> interrupts per second, especially if multiple interfaces are running
> under load, it might be important. And the 750x and 752x chipsets are
> so common that it is worthwhile to deal with them (and there are reports
> that the aliasing problem is happening on more chipsets than just these
> now).
I think you've sold me.
I'm in the process of trying to justify time to write a FreeBSD driver
for our PCIe 10GbE cards, and I find what you're doing fascinating.
I'd like to use some of your techniques for the driver I'm writing.
> As for latency, the taskqueue runs at the same PI_NET priority as an the
> ithread would. I thought that there was an optimization on some
> platforms to encourage quick preemption for ithreads when they are
> scheduled, but I can't find it now. So, the taskqueue shouldn't be all
> that different from an ithread, and it even means that there won't be
> any sharing between instances even if the interrupt vector is shared.
OK that (a taskqueue not getting the same fast preemption an ithread
would) was just what I was afraid of. I'm glad that it is not a
problem.
The only problems I saw here were early on when I had the taskqueue
running at a normal thread priority. It was constantly being preempted
by the netisr, resulting in really bad performance. Moving it up to
ithread-equivalent priority fixed this. If you copy this, make sure
you pay close attention to the sched_prio() call that is made in the
driver. I don't like exposing the driver to the low-level scheduling
interface, so the long term plan is to either wrap it into the taskqueue
API, or implement the two-stage interrupt API.
<...>
> However, taskqueues are really just a proof of concept for what I really
> want, which is to allow drivers to register both a fast handler and an
> ithread handler. For drivers doing this, the ithread would be private
Ah, the darwin / MacOSX model. That would be very cool.
Yep. Working in IOKit was very interesting, and this is one of the few
things that transfers well to FreeBSD. C++ does have a certain elagence
for drivers, but the cost of virtual methods in the fast path of the
driver and stack is still far too high to justify using it.
Anyway, keep up the good work. It is inspiring!
Thanks!
Scott
_______________________________________________
cvs-all@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/cvs-all
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
