On Thu, 28 Jul 2016 00:18:14 +1000 Michael Ellerman <m...@ellerman.id.au> wrote:
> From: Kevin Hao <haoke...@gmail.com> > > We do binary patching of asm code using CPU features, which is a > one-time operation, done during early boot. However checks of CPU > features in C code are currently done at run time, even though the set > of CPU features can never change after boot. > > We can optimise this by using jump labels to implement > cpu_has_feature(), meaning checks in C code are binary patched into a > single nop or branch. > > For a C sequence along the lines of: > > if (cpu_has_feature(FOO)) > return 2; > > The generated code before is roughly: > > ld r9,-27640(r2) > ld r9,0(r9) > lwz r9,32(r9) > cmpwi cr7,r9,0 > bge cr7, 1f > li r3,2 > blr > 1: ... > > After (true): > nop > li r3,2 > blr > > After (false): > b 1f > li r3,2 > blr > 1: ... > > Signed-off-by: Kevin Hao <haoke...@gmail.com> > Signed-off-by: Aneesh Kumar K.V <aneesh.ku...@linux.vnet.ibm.com> > Signed-off-by: Michael Ellerman <m...@ellerman.id.au> > --- > arch/powerpc/include/asm/cpu_has_feature.h | 22 > ++++++++++++++++++++++ arch/powerpc/include/asm/cputable.h | > 6 ++++++ arch/powerpc/kernel/cputable.c | 20 > ++++++++++++++++++++ arch/powerpc/lib/feature-fixups.c | 1 + > 4 files changed, 49 insertions(+) > > v3: Rename MAX_CPU_FEATURES as we already have a #define with that > name. Define NUM_CPU_FTR_KEYS as a constant. > Rename the array to cpu_feature_keys. > Use the kconfig we added to guard it. > Rewrite the change log. > > diff --git a/arch/powerpc/include/asm/cpu_has_feature.h > b/arch/powerpc/include/asm/cpu_has_feature.h index > ad296b2f1d84..18e60e61bea9 100644 --- > a/arch/powerpc/include/asm/cpu_has_feature.h +++ > b/arch/powerpc/include/asm/cpu_has_feature.h @@ -11,10 +11,32 @@ > static inline bool __cpu_has_feature(unsigned long feature) > (CPU_FTRS_POSSIBLE & cur_cpu_spec->cpu_features & feature)); } > > +#ifdef CONFIG_JUMP_LABEL_FEATURE_CHECKS > +#include <linux/jump_label.h> > + > +#define NUM_CPU_FTR_KEYS 64 > + > +extern struct static_key_true cpu_feature_keys[NUM_CPU_FTR_KEYS]; > + > +static __always_inline bool cpu_has_feature(unsigned long feature) > +{ > + int i; > + > + if (CPU_FTRS_ALWAYS & feature) > + return true; > + > + if (!(CPU_FTRS_POSSIBLE & feature)) > + return false; > + > + i = __builtin_ctzl(feature); > + return static_branch_likely(&cpu_feature_keys[i]); Just a reminder to add a BUILD_BUG_ON(!__builtin_constant_p(feature)); for this. _______________________________________________ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev