On Mon, 19 Oct 2020 at 15:04, Marc Zyngier <m...@kernel.org> wrote:
>
> Hi Vincent,
>
> On 2020-10-19 13:42, Vincent Guittot wrote:
> > Hi Marc,
> >
> > On Tue, 1 Sep 2020 at 16:44, Marc Zyngier <m...@kernel.org> wrote:
> >>
> >> In order to deal with IPIs as normal interrupts, let's add
> >> a new way to register them with the architecture code.
> >>
> >> set_smp_ipi_range() takes a range of interrupts, and allows
> >> the arch code to request them as if the were normal interrupts.
> >> A standard handler is then called by the core IRQ code to deal
> >> with the IPI.
> >>
> >> This means that we don't need to call irq_enter/irq_exit, and
> >> that we don't need to deal with set_irq_regs either. So let's
> >> move the dispatcher into its own function, and leave handle_IPI()
> >> as a compatibility function.
> >>
> >> On the sending side, let's make use of ipi_send_mask, which
> >> already exists for this purpose.
> >>
> >> One of the major difference is that we end up, in some cases
> >> (such as when performing IRQ time accounting on the scheduler
> >> IPI), end up with nested irq_enter()/irq_exit() pairs.
> >> Other than the (relatively small) overhead, there should be
> >> no consequences to it (these pairs are designed to nest
> >> correctly, and the accounting shouldn't be off).
> >
> > While rebasing on mainline, I have faced a performance regression for
> > the benchmark:
> > perf bench sched pipe
> > on my arm64 dual quad core (hikey) and my 2 nodes x 112 CPUS (thx2)
> >
> > The regression comes from:
> > commit: d3afc7f12987 ("arm64: Allow IPIs to be handled as normal
> > interrupts")
>
> That's interesting, as this patch doesn't really change anything (most
> of the potential overhead comes in later). The only potential overhead
> I can see is that the scheduler_ipi() call is now wrapped around
> irq_enter()/irq_exit().
>
> >
> > v5.9 + this patch
> > hikey : 48818(+/- 0.31) 37503(+/- 0.15%) -23.2%
> > thx2 : 132410(+/- 1.72) 122646(+/- 1.92%) -7.4%
> >
> > By + this patch, I mean merging branch from this patch. Whereas
> > merging the previous:
> > commit: 83cfac95c018 ("genirq: Allow interrupts to be excluded from
> > /proc/interrupts")
> > It doesn't show any regression
>
> Since you are running perf, can you spot where the overhead occurs?
hmm... Difficult to say because tracing the bench decreases a lot the
result. I have pasted the perf reports.
With this patch :
# Samples: 634 of event 'cpu-clock'
# Event count (approx.): 158500000
#
# Overhead Command Shared Object Symbol
# ........ .......... .................. ..................................
#
31.86% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
8.68% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irq
6.31% sched-pipe [kernel.kallsyms] [k] __schedule
5.21% sched-pipe [kernel.kallsyms] [k] schedule
4.73% sched-pipe [kernel.kallsyms] [k] pipe_read
3.31% sched-pipe [kernel.kallsyms] [k] el0_svc_common.constprop.3
2.84% sched-pipe [kernel.kallsyms] [k] ww_mutex_lock_interruptible
2.52% sched-pipe [kernel.kallsyms] [k] init_wait_entry
2.37% sched-pipe [kernel.kallsyms] [k] mutex_unlock
2.21% sched-pipe [kernel.kallsyms] [k] new_sync_read
1.89% sched-pipe [kernel.kallsyms] [k] new_sync_write
1.74% sched-pipe [kernel.kallsyms] [k] security_file_permission
1.74% sched-pipe [kernel.kallsyms] [k] vfs_read
1.58% sched-pipe [kernel.kallsyms] [k] __my_cpu_offset
1.26% sched-pipe libpthread-2.24.so [.] 0x0000000000010a2c
1.10% sched-pipe [kernel.kallsyms] [k] mutex_lock
1.10% sched-pipe [kernel.kallsyms] [k] vfs_write
After reverting this patch which gives a result similar to v5.9:
# Samples: 659 of event 'cpu-clock'
# Event count (approx.): 164750000
#
# Overhead Command Shared Object Symbol
# ........ .......... .................. ...............................
#
29.29% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
21.40% sched-pipe [kernel.kallsyms] [k] _raw_spin_unlock_irq
4.86% sched-pipe [kernel.kallsyms] [k] pipe_read
4.55% sched-pipe [kernel.kallsyms] [k] ww_mutex_lock_interruptible
2.88% sched-pipe [kernel.kallsyms] [k] __schedule
2.88% sched-pipe [kernel.kallsyms] [k] _raw_spin_lock_irqsave
2.88% sched-pipe [kernel.kallsyms] [k] schedule
2.12% sched-pipe [kernel.kallsyms] [k] new_sync_read
1.82% sched-pipe [kernel.kallsyms] [k] mutex_lock
1.67% sched-pipe [kernel.kallsyms] [k] el0_svc_common.constprop.3
1.67% sched-pipe [kernel.kallsyms] [k] pipe_write
1.21% sched-pipe [kernel.kallsyms] [k] rw_verify_area
1.21% sched-pipe [kernel.kallsyms] [k] security_file_permission
1.06% sched-pipe [kernel.kallsyms] [k] fsnotify
I have only put symbol with overhead above 1%
so _raw_spin_unlock_irq, schedule and __schedule seem the most
impacted but i can't get any conclusion
I can sent you perf.data files if you want
>
> Thanks,
>
> M.
> --
> Jazz is not dead. It just smells funny...
