johan...@sun.com wrote:
On Fri, Jun 19, 2009 at 05:52:02PM +0200, Nils Goroll wrote:
I am trying to reduce context switches on a V890 with 16 cores by
delegating one core to interrupt handling (by setting all others to
nointr using psradm).
The best way to do this is to create processor sets, with the set that
handles OS tasks containing processors that handle interrupts and the
other set containining the nointr processors. If you're really
concerned about context switching, you might also benefit from processor
binding, which only lets a process run on a particular cpu.
Lately, the scheduler guys have made a bunch of improvements that allow
processes to be more sticky, though. (Less likely to migrate from one
CPU to another, but that is different from a context switch) What
version of the OS are you running?
Does the hardware design of this machine imply that (a) particular
core(s) is best suited for this task?
Almost any multiprocessor machine that has NUMA characteristics will
benefit from having the interrupts placed on the CPU that's closest to
the hardware that connects to the bus that the interrupts are coming
from. From a discussion long ago, I remember the v890 has some NUMA
characteristics, but I don't honestly know if we treat it like a NUMA
machine. Jonathan Chew, who's lurking on the list somewhere, might have
more details.
Lurking?! Who me? I'm innocent. ;-)
I think that Paul already replied and explained that the V890 doesn't
have NUMA I/O, so it doesn't matter which CPU is used for the interrupt
with respect to the device for NUMA. However, it may matter where you
assign the interrupt for NUMA memory locality and CMT cache sharing
(since each UltraSPARC IV+ processor chip has two single stranded cores
that share cache)....
If there are (user or kernel) threads that are involved in doing I/O
to/from the device, they may benefit from sharing (or not sharing) cache
and/or local memory with the interrupt thread and CPU where the
interrupt is assigned. For example, the Crossbow poll thread will
benefit from being bound to the same CPU as the interrupt or the other
CPU sharing cache with the interrupt thread/CPU and the Crossbow soft
ring threads will at least benefit from sharing local memory with the
interrupt thread/CPU if you're doing network I/O on OpenSolaris. You
also should be careful what threads are placed on the CPU sharing cache
with the interrupt CPU and thread because the threads may interfere with
each other's cache utilization.
Your user application threads may benefit from running close or far away
from the interrupt thread/CPU depending on what the threads do.
Thus, how you place your interrupt and your threads and whether you use
processor sets, processor binding, etc. will depend on your workload and
what you are trying to optimize.
Hope that helps.
Jonathan
PS
Now I'll go back to lurking/sleeping. ;-)
_______________________________________________
perf-discuss mailing list
perf-discuss@opensolaris.org
