Commit f77084d96355 "x86/mm/pat: Disable preemption around __flush_tlb_all()" introduced a warning to capture cases __flush_tlb_all() is called without pre-emption disabled. It triggers a false positive warning in the memory hotplug path. On investigation it was found that the __flush_tlb_all() calls are not necessary. However, they are only "not necessary" in practice provided the ptes are being initially populated from the !present state. Introduce set_pte_safe() as a sanity check that the pte is being updated in a way that does not require a tlb flush.
Forgive the macro, the availability of the various of set_pte() levels is hit and miss across architectures. Link: https://lore.kernel.org/lkml/279dadae-9148-465c-7ec6-3f37e026c...@intel.com Cc: Kirill A. Shutemov <kirill.shute...@linux.intel.com> Cc: Sebastian Andrzej Siewior <bige...@linutronix.de> Cc: Thomas Gleixner <t...@linutronix.de> Cc: Peter Zijlstra <pet...@infradead.org> Cc: Borislav Petkov <b...@alien8.de> Suggested-by: Peter Zijlstra <pet...@infradead.org> Suggested-by: Dave Hansen <dave.han...@intel.com> Signed-off-by: Dan Williams <dan.j.willi...@intel.com> --- include/asm-generic/pgtable.h | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index dae7f98babed..62be0d5e1a9a 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -400,6 +400,41 @@ static inline int pgd_same(pgd_t pgd_a, pgd_t pgd_b) } #endif +/* + * The _safe versions of set_{pte,pmd,pud,p4d,pgd} validate that the + * entry was not populated previously. I.e. for cases where a flush-tlb + * is elided, double-check that there is no stale mapping to shoot down. + */ +#define set_pte_safe(ptep, pte) \ +({ \ + WARN_ON_ONCE(pte_present(*ptep) && !pte_same(*ptep, pte)); \ + set_pte(ptep, pte); \ +}) + +#define set_pmd_safe(pmdp, pmd) \ +({ \ + WARN_ON_ONCE(pmd_present(*pmdp) && !pmd_same(*pmdp, pmd)); \ + set_pmd(pmdp, pmd); \ +}) + +#define set_pud_safe(pudp, pud) \ +({ \ + WARN_ON_ONCE(pud_present(*pudp) && !pud_same(*pudp, pud)); \ + set_pud(pudp, pud); \ +}) + +#define set_p4d_safe(p4dp, p4d) \ +({ \ + WARN_ON_ONCE(p4d_present(*p4dp) && !p4d_same(*p4dp, p4d)); \ + set_p4d(p4dp, p4d); \ +}) + +#define set_pgd_safe(pgdp, pgd) \ +({ \ + WARN_ON_ONCE(pgd_present(*pgdp) && !pgd_same(*pgdp, pgd)); \ + set_pgd(pgdp, pgd); \ +}) + #ifndef __HAVE_ARCH_DO_SWAP_PAGE /* * Some architectures support metadata associated with a page. When a
