Convert alloc_anon_folio() to pass __GFP_ZERO instead of zeroing at the callsite. post_alloc_hook uses the fault address passed through vma_alloc_folio for cache-friendly zeroing.
Note: before this series, replacing clear_user_highpage() with __GFP_ZERO was unsafe on cache-aliasing architectures because __GFP_ZERO uses clear_page() without a dcache flush. With this series, it is safe if the caller passes a valid user address (not USER_ADDR_NONE) to vma_alloc_folio() etc., which delivers it to post_alloc_hook() for the dcache flush via folio_zero_user(). It is only unsafe if USER_ADDR_NONE is passed. Note: with __GFP_ZERO, the folio is zeroed before mem_cgroup_charge(). If the charge fails, the zeroing work is wasted. Previously zeroing was done after a successful charge. This is inherent to moving zeroing into the allocator. Charge failures are rare (only at cgroup limits). Use folio_put_zeroed() on charge failure so the zeroed hint propagates to the buddy allocator, avoiding redundant re-zeroing on the next allocation attempt. Signed-off-by: Michael S. Tsirkin <[email protected]> Reviewed-by: Gregory Price <[email protected]> Assisted-by: Claude:claude-opus-4-6 --- mm/memory.c | 13 ++----------- 1 file changed, 2 insertions(+), 11 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 6c14b90f558e..6d6a3e1a02c1 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5265,25 +5265,16 @@ static struct folio *alloc_anon_folio(struct vm_fault *vmf) goto fallback; /* Try allocating the highest of the remaining orders. */ - gfp = vma_thp_gfp_mask(vma); + gfp = vma_thp_gfp_mask(vma) | __GFP_ZERO; while (orders) { folio = vma_alloc_folio(gfp, order, vma, vmf->address); if (folio) { if (mem_cgroup_charge(folio, vma->vm_mm, gfp)) { count_mthp_stat(order, MTHP_STAT_ANON_FAULT_FALLBACK_CHARGE); - folio_put(folio); + folio_put_zeroed(folio); goto next; } folio_throttle_swaprate(folio, gfp); - /* - * When a folio is not zeroed during allocation - * (__GFP_ZERO not used) or user folios require special - * handling, folio_zero_user() is used to make sure - * that the page corresponding to the faulting address - * will be hot in the cache after zeroing. - */ - if (user_alloc_needs_zeroing()) - folio_zero_user(folio, vmf->address); return folio; } next: -- MST
