Hi Steve, Arnd,
On Tue, Dec 04, 2018 at 12:50:12AM -0500, Steven Rostedt wrote:
> On Mon, 3 Dec 2018 22:51:52 +0100
> Arnd Bergmann <a...@arndb.de> wrote:
> > On Mon, Dec 3, 2018 at 8:22 PM Will Deacon <will.dea...@arm.com> wrote:
> > > On Fri, Nov 30, 2018 at 04:09:56PM +0100, Anders Roxell wrote:
> > > > Both of those functions end up calling ftrace_modify_code(), which is
> > > > expensive because it changes the page tables and flush caches.
> > > > Microseconds add up because this is called in a loop for each dyn_ftrace
> > > > record, and this triggers the softlockup watchdog unless we let it sleep
> > > > occasionally.
> > > > Rework so that we call cond_resched() before going into the
> > > > ftrace_modify_code() function.
> > > >
> > > > Co-developed-by: Arnd Bergmann <a...@arndb.de>
> > > > Signed-off-by: Arnd Bergmann <a...@arndb.de>
> > > > Signed-off-by: Anders Roxell <anders.rox...@linaro.org>
> > > > ---
> > > > arch/arm64/kernel/ftrace.c | 10 ++++++++++
> > > > 1 file changed, 10 insertions(+)
> > >
> > > It sounds like you're running into issues with the existing code, but I'd
> > > like to understand a bit more about exactly what you're seeing. Which part
> > > of the ftrace patching is proving to be expensive?
> > >
> > > The page table manipulation only happens once per module when using PLTs,
> > > and the cache maintenance is just a single line per patch site without an
> > > IPI.
> > >
> > > Is it the loop in ftrace_replace_code() that is causing the hassle?
> >
> > Yes: with an allmodconfig kernel, the ftrace selftest calls
> > ftrace_replace_code
> > to look >40000 through ftrace_make_call/ftrace_make_nop, and these
> > end up calling
Ok, 40000 invocations would do it!
> > static int __kprobes __aarch64_insn_write(void *addr, __le32 insn)
> > {
> > void *waddr = addr;
> > unsigned long flags = 0;
> > int ret;
> >
> > raw_spin_lock_irqsave(&patch_lock, flags);
> > waddr = patch_map(addr, FIX_TEXT_POKE0);
> >
> > ret = probe_kernel_write(waddr, &insn, AARCH64_INSN_SIZE);
> >
> > patch_unmap(FIX_TEXT_POKE0);
> > raw_spin_unlock_irqrestore(&patch_lock, flags);
> >
> > return ret;
> > }
> > int __kprobes aarch64_insn_patch_text_nosync(void *addr, u32 insn)
> > {
> > u32 *tp = addr;
> > int ret;
> >
> > /* A64 instructions must be word aligned */
> > if ((uintptr_t)tp & 0x3)
> > return -EINVAL;
> >
> > ret = aarch64_insn_write(tp, insn);
> > if (ret == 0)
> > __flush_icache_range((uintptr_t)tp,
> > (uintptr_t)tp + AARCH64_INSN_SIZE);
> >
> > return ret;
> > }
> >
> > which seems to be where the main cost is. This is with inside of
> > qemu, and with lots of debugging options (in particular
> > kcov and ubsan) enabled, that make each function call
> > more expensive.
>
> I was thinking more about this. Would something like this work?
>
> -- Steve
>
> diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c
> index 8ef9fc226037..42e89397778b 100644
> --- a/kernel/trace/ftrace.c
> +++ b/kernel/trace/ftrace.c
> @@ -2393,11 +2393,14 @@ void __weak ftrace_replace_code(int enable)
> {
> struct dyn_ftrace *rec;
> struct ftrace_page *pg;
> + bool schedulable;
> int failed;
>
> if (unlikely(ftrace_disabled))
> return;
>
> + schedulable = !irqs_disabled() & !preempt_count();
Looks suspiciously like a bitwise preemptible() to me!
> +
> do_for_each_ftrace_rec(pg, rec) {
>
> if (rec->flags & FTRACE_FL_DISABLED)
> @@ -2409,6 +2412,8 @@ void __weak ftrace_replace_code(int enable)
> /* Stop processing */
> return;
> }
> + if (schedulable)
> + cond_resched();
> } while_for_each_ftrace_rec();
> }
If this solves the problem in core code, them I'm all for it. Otherwise, I
was thinking of rolling our own ftrace_replace_code() for arm64, but that's
going to involve a fair amount of duplication.
Will
