On Fri, 3 Feb 2017 03:20:48 PM Nicholas Piggin wrote:
> On Fri, 3 Feb 2017 00:25:11 +1000
> Nicholas Piggin <npig...@gmail.com> wrote:
>
> > This goes with the previous NMI IPI series, and a new version of
> > Alistair's opal API I posted to the skiboot list.
>
> And here is the incremental bit that is required for Alistair's
> hardware implementation to work.
>
> If the opal broacast call fails with OPAL_PARTIAL, then we designate
> a bouncer CPU on another core to send NMI IPIs back to our sibling
> threads.
>
> Probably needs more discussion and testing about how to detect and
> handle failure cases and future compatibility for different types of
> restrictions, but at least it works in mambo.
Looking at the Mambo implementation you recently posted I couldn't see
where OPAL_PARTIAL was returned. So I guess you have other Skiboot
patches to test this path using Mambo? I'm wondering because they
could be useful for testing the skiboot HW side as well...
>
> Of course the other option rather than doing this in Linux is call
> into opal in the system reset handler and have it do the bouncing.
> Something to consider before we finalise the API.
That might not be such a bad idea. OPAL could queue up the threads it
couldn't reset and then wait until opal_sreset_complete() is called
from an eligible thread to reset the ones it couldn't do in the
original call.
I might try prototyping something like this when I get some time. One
issue would be if there is only a single core in the system, but
that's unlikely and I think that's probably something we can't support
in any case as cores can't sreset threads on the same core, at least
on P8.
- Alistair
> Thanks,
> Nick
>
> ---
> arch/powerpc/platforms/powernv/smp.c | 88
> +++++++++++++++++++++++++++++++++---
> 1 file changed, 82 insertions(+), 6 deletions(-)
>
> diff --git a/arch/powerpc/platforms/powernv/smp.c
> b/arch/powerpc/platforms/powernv/smp.c
> index f90555f75723..4241fda9df9e 100644
> --- a/arch/powerpc/platforms/powernv/smp.c
> +++ b/arch/powerpc/platforms/powernv/smp.c
> @@ -241,8 +241,32 @@ static int pnv_cpu_bootable(unsigned int nr)
> return smp_generic_cpu_bootable(nr);
> }
>
> +static int nmi_ipi_bounce_cpu;
> +static int nmi_ipi_bounce_cpu_done;
> +static int nmi_ipi_bounce_target_core;
> +static int nmi_ipi_bounce_target_exclude;
> +
> int pnv_system_reset_exception(struct pt_regs *regs)
> {
> + smp_rmb();
> + if (nmi_ipi_bounce_cpu == smp_processor_id()) {
> + int64_t rc;
> + int c;
> +
> + for_each_online_cpu(c) {
> + if (!cpumask_test_cpu(c,
> cpu_sibling_mask(nmi_ipi_bounce_target_core)))
> + continue;
> + if (c == nmi_ipi_bounce_target_exclude)
> + continue;
> + rc = opal_signal_system_reset(c);
> + if (rc != OPAL_SUCCESS) {
> + nmi_ipi_bounce_cpu_done = -1;
> + return 1;
> + }
> + }
> + nmi_ipi_bounce_cpu_done = 1;
> + }
> +
> if (smp_handle_nmi_ipi(regs))
> return 1;
> return 0;
> @@ -252,13 +276,65 @@ static int pnv_cause_nmi_ipi(int cpu)
> {
> int64_t rc;
>
> - rc = opal_signal_system_reset(cpu);
> - if (rc == OPAL_SUCCESS)
> - return 1;
> + if (cpu >= 0) {
> + rc = opal_signal_system_reset(cpu);
> + if (rc == OPAL_SUCCESS)
> + return 1;
> + } else {
> + /*
> + * Test bounce behavior with broadcast IPI.
> + */
> + rc = OPAL_PARTIAL;
> + }
> + if (rc == OPAL_PARTIAL) {
> + int c;
>
> - /*
> - * Don't cope with OPAL_PARTIAL yet (just punt to regular IPI)
> - */
> + /*
> + * Some platforms can not send NMI to sibling threads in
> + * the same core. We can designate one inter-core target
> + * to bounce NMIs back to our sibling threads.
> + */
> +
> + if (cpu >= 0) {
> + /*
> + * Don't support bouncing unicast NMIs yet (because
> + * that would have to raise an NMI on an unrelated
> + * CPU. Revisit this if callers start using unicast.
> + */
> + return 0;
> + }
> +
> + nmi_ipi_bounce_cpu = -1;
> + nmi_ipi_bounce_cpu_done = 0;
> + nmi_ipi_bounce_target_core = -1;
> + nmi_ipi_bounce_target_exclude = -1;
> + smp_wmb();
> +
> + for_each_online_cpu(c) {
> + if (cpumask_test_cpu(c,
> cpu_sibling_mask(smp_processor_id())))
> + continue;
> +
> + if (nmi_ipi_bounce_cpu == -1) {
> + nmi_ipi_bounce_cpu = c;
> + nmi_ipi_bounce_target_core = smp_processor_id();
> + if (cpu == NMI_IPI_ALL_OTHERS)
> + nmi_ipi_bounce_target_exclude =
> smp_processor_id();
> + }
> +
> + rc = opal_signal_system_reset(c);
> + if (rc != OPAL_SUCCESS)
> + return 0;
> + }
> +
> + if (nmi_ipi_bounce_cpu == -1)
> + return 0; /* could not find a bouncer */
> +
> + while (!nmi_ipi_bounce_cpu_done)
> + cpu_relax();
> +
> + if (nmi_ipi_bounce_cpu_done == 1)
> + return 1; /* bounce worked */
> + }
>
> return 0;
> }
>
