On Wednesday, May 02, 2012 5:16:17 pm Navdeep Parhar wrote:
> There seems to be a regression in 8.3 in the way the kernel selects CPUs
> for interrupts. For example, cxgb(4) on 8.3 ends up with all
> its ithreads on the same CPU (CPU7 in this case).
>
> 12 root -68 - 0K 816K WAIT 7 0:55 0.00% intr{irq279:
> cxgbc0}
> 12 root -68 - 0K 816K WAIT 7 0:52 0.00% intr{irq275:
> cxgbc0}
> 12 root -68 - 0K 816K WAIT 7 0:47 0.00% intr{irq278:
> cxgbc0}
> 12 root -68 - 0K 816K WAIT 7 0:43 0.00% intr{irq277:
> cxgbc0}
> 12 root -68 - 0K 816K WAIT 7 0:43 0.00% intr{irq282:
> cxgbc0}
> 12 root -68 - 0K 816K WAIT 7 0:41 0.00% intr{irq281:
> cxgbc0}
> 12 root -68 - 0K 816K WAIT 7 0:32 0.00% intr{irq276:
> cxgbc0}
> 12 root -68 - 0K 816K WAIT 7 0:31 0.00% intr{irq280:
> cxgbc0}
>
> Back in the day there used to be code in cxgb to bind different
> interrupts to different CPUs but it was removed because the kernel
> distributed them across CPUs anyway. So what changed? This appears 8.3
> specific. I don't see it on head and I don't have a 9 system readily
> available right now.
Hmmm, that seems odd. It is true that the round-robin that the OS does only
pins the low-level message from the APIC/MSI vector to the CPU. It does not
affect the thread. However, ULE prefers to run ithreads on the CPU that
processes the interrupt:
static int
sched_pickcpu(struct thread *td, int flags)
{
...
/*
* Prefer to run interrupt threads on the processors that generate
* the interrupt.
*/
if (td->td_priority <= PRI_MAX_ITHD && THREAD_CAN_SCHED(td, self) &&
curthread->td_intr_nesting_level && ts->ts_cpu != self) {
SCHED_STAT_INC(pickcpu_intrbind);
ts->ts_cpu = self;
}
}
--
John Baldwin
_______________________________________________
[email protected] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[email protected]"
