From: Arvind Yadav <[email protected]>
__migrate_device_pages() reads the folio mapping before calling
folio_free_swap(). When folio_free_swap() succeeds, the folio is removed
from the swap cache, but the saved mapping still points to swap_space.
Passing the stale mapping to folio_migrate_mapping() makes it use the
mapped-folio path for a folio that is no longer in swapcache. It can
then operate on swap_space.i_pages with invalid reference accounting,
eventually triggering a folio reference count BUG.
After a successful split, nr still contains the number of pages in the
original large folio, although each resulting page is now a separate
order-0 folio. Reset nr to 1 so each split folio is processed separately,
including its own swapcache removal and mapping lookup.
Refresh the saved mapping after folio_free_swap() so the current folio
state is used during migration.
v2:
- Refresh the mapping using folio_mapping(), as suggested by Zi Yan.
v3:
- Reset nr to 1 after a successful split so each resulting folio is
processed independently, as suggested by Zi Yan.
v4:
- Re-read each source folio's mapping immediately before
folio_migrate_mapping(), as suggested by Balbir Singh.
- Add a warning to validate the post-split order-0 invariant,
as suggested by Balbir Singh.
Fixes: df263d9a7dff ("mm/migrate_device: try to handle swapcache pages")
Cc: Andrew Morton <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: Matthew Brost <[email protected]>
Cc: Joshua Hahn <[email protected]>
Cc: Rakie Kim <[email protected]>
Cc: Byungchul Park <[email protected]>
Cc: Gregory Price <[email protected]>
Cc: Ying Huang <[email protected]>
Cc: Alistair Popple <[email protected]>
Reviewed-by: Zi Yan <[email protected]>
Reviewed-by: Balbir Singh <[email protected]>
Signed-off-by: Arvind Yadav <[email protected]>
---
mm/migrate_device.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/mm/migrate_device.c b/mm/migrate_device.c
index 908d2d4ec43a..162d29b2807a 100644
--- a/mm/migrate_device.c
+++ b/mm/migrate_device.c
@@ -1183,6 +1183,13 @@ static void __migrate_device_pages(unsigned long
*src_pfns,
MIGRATE_PFN_COMPOUND);
goto next;
}
+
+ /*
+ * reset nr so that only first after-split folio
+ * is processed below
+ */
+ VM_WARN_ON_ONCE(folio_test_large(folio));
+ nr = 1;
} else if ((src_pfns[i] & MIGRATE_PFN_MIGRATE) &&
(dst_pfns[i] & MIGRATE_PFN_COMPOUND) &&
!(src_pfns[i] & MIGRATE_PFN_COMPOUND)) {
@@ -1222,6 +1229,12 @@ static void __migrate_device_pages(unsigned long
*src_pfns,
folio = page_folio(migrate_pfn_to_page(src_pfns[i+j]));
newfolio =
page_folio(migrate_pfn_to_page(dst_pfns[i+j]));
+ /*
+ * folio_free_swap() removed the folio from the swap
+ * cache. Refresh the saved mapping before migration.
+ */
+ mapping = folio_mapping(folio);
+
r = folio_migrate_mapping(mapping, newfolio, folio,
extra_cnt);
if (r)
src_pfns[i+j] &= ~MIGRATE_PFN_MIGRATE;
--
2.34.1
