From: Barry Song <[email protected]> If the current do_swap_page() took the per-VMA lock and we dropped it only to wait for I/O completion (e.g., use folio_wait_locked()), then when do_swap_page() is retried after the I/O completes, it should still qualify for the per-VMA-lock path.
Cc: David Hildenbrand <[email protected]> Cc: Lorenzo Stoakes <[email protected]> Cc: Liam R. Howlett <[email protected]> Cc: Vlastimil Babka <[email protected]> Cc: Mike Rapoport <[email protected]> Cc: Suren Baghdasaryan <[email protected]> Cc: Michal Hocko <[email protected]> Cc: Paul Walmsley <[email protected]> Cc: Palmer Dabbelt <[email protected]> Cc: Albert Ou <[email protected]> Cc: Alexandre Ghiti <[email protected]> Cc: Russell King <[email protected]> Cc: Catalin Marinas <[email protected]> Cc: Will Deacon <[email protected]> Cc: Huacai Chen <[email protected]> Cc: WANG Xuerui <[email protected]> Cc: Madhavan Srinivasan <[email protected]> Cc: Michael Ellerman <[email protected]> Cc: Nicholas Piggin <[email protected]> Cc: Christophe Leroy <[email protected]> Cc: Alexander Gordeev <[email protected]> Cc: Gerald Schaefer <[email protected]> Cc: Heiko Carstens <[email protected]> Cc: Vasily Gorbik <[email protected]> Cc: Christian Borntraeger <[email protected]> Cc: Sven Schnelle <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Cc: Ingo Molnar <[email protected]> Cc: Borislav Petkov <[email protected]> Cc: [email protected] Cc: H. Peter Anvin <[email protected]> Cc: Matthew Wilcox <[email protected]> Cc: Pedro Falcato <[email protected]> Cc: Jarkko Sakkinen <[email protected]> Cc: Oscar Salvador <[email protected]> Cc: Kuninori Morimoto <[email protected]> Cc: Oven Liyang <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Ada Couprie Diaz <[email protected]> Cc: Robin Murphy <[email protected]> Cc: Kristina Martšenko <[email protected]> Cc: Kevin Brodsky <[email protected]> Cc: Yeoreum Yun <[email protected]> Cc: Wentao Guan <[email protected]> Cc: Thorsten Blum <[email protected]> Cc: Steven Rostedt <[email protected]> Cc: Yunhui Cui <[email protected]> Cc: Nam Cao <[email protected]> Cc: Chris Li <[email protected]> Cc: Kairui Song <[email protected]> Cc: Kemeng Shi <[email protected]> Cc: Nhat Pham <[email protected]> Cc: Baoquan He <[email protected]> Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Cc: [email protected] Signed-off-by: Barry Song <[email protected]> --- mm/memory.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 4f933fedd33e..7f70f0324dcf 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4654,6 +4654,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) unsigned long page_idx; unsigned long address; pte_t *ptep; + bool retry_by_vma_lock = false; if (!pte_unmap_same(vmf)) goto out; @@ -4758,8 +4759,13 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) swapcache = folio; ret |= folio_lock_or_retry(folio, vmf); - if (ret & VM_FAULT_RETRY) + if (ret & VM_FAULT_RETRY) { + if (fault_flag_allow_retry_first(vmf->flags) && + !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT) && + (vmf->flags & FAULT_FLAG_VMA_LOCK)) + retry_by_vma_lock = true; goto out_release; + } page = folio_file_page(folio, swp_offset(entry)); /* @@ -5044,7 +5050,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) } if (si) put_swap_device(si); - return ret; + return ret | (retry_by_vma_lock ? VM_FAULT_RETRY_VMA : 0); } static bool pte_range_none(pte_t *pte, int nr_pages) -- 2.39.3 (Apple Git-146)
