On May 13, 2008, at 5:05 PM, Corey Ashford wrote:
Hello,
One of the things I've been working on is porting perfmon2 to ppc64.
We've made a fair amount of progress on it, and support is available
in libpfm and the perfmon2 kernel patch.
One of the things we had to work around was the "lazy interrupt
disabling" mechanism in ppc64 Linux. The problem was that the PMU
exception handler (0xf00) uses the STD_EXCEPTION_PSERIES wrapper,
which does not support lazy interrupt disabling.
This is desirable for Oprofile's use of the PMU since its handler is
fairly simple and being able to profile interrupt protected code is
desirable. However, it causes problems for perfmon2, since the
operations it performs on the thread of its PMU interrupt handler
can cause a deadlock condition (it can end up calling spin_lock, for
example).
Initially, to work around this, we created special spin_lock_irqsave
and spin_unlock_irqrestore macros for perfmon2 which we could
override for POWER to define them as functions which do hard
disables and restores.
However, not all of the places that we need to disable interrupts
were occurring from within the perfmon2 code. Specifically, getting
PMU interrupts in the middle of a schedule() call (where interrupts
were expected to be disabled) was causing kernel hangs.
To fix this, I've gone back and removed the special spin_lock macros
we defined in perfmon2 and have ifdef'd arch/powerpc/kernel/
head_64.S file as follows:
/*** pSeries interrupt support ***/
/* moved from 0xf00 */
+ #ifdef CONFIG_PERFMON
+ MASKABLE_EXCEPTION_PSERIES(., performance_monitor)
+ #else
STD_EXCEPTION_PSERIES(., performance_monitor)
+ #endif
/*
* An interrupt came in while soft-disabled; clear EE in SRR1,
* clear paca->hard_enabled and return.
The downside of this change is that if someone is using Oprofile and
they have enabled perfmon in their kernel, they will not get profile
samples that occur in interrupt-protected regions of the kernel.
However, they still can by configuring perfmon out of their kernel.
What do you think of this idea? Is this something that you would
object to when perfmon2 does go upstream to LKML? I think we'd want
to add some documentation somewhere that describes this side-effect
of enabling perfmon in the ppc64 Linux kernel. Do you have
suggestions on where that should go? I'm thinking perhaps in both
the basic_profiling.txt and perfmon2.txt files in the Documentation
subdirectory.
Since you didn't post the perfmon2 code, I'll ask instead of go look:
Do you have a single entry point from performance_monitor into
perfmon2? If so, it's nicer to check and see if interrupts are soft
disabled right upon entry (before taking any locks, etc), and if they
are just return without doing more work.
PMU interrupts generally won't re-arm themselves so you'll obviously
have to deal with that as well but I'm sure you're already aware of
that...
-Olof
_______________________________________________
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
