** Also affects: linux (Ubuntu Noble)
Importance: Undecided
Status: New
** Also affects: linux-nvidia (Ubuntu Noble)
Importance: Undecided
Status: New
** Also affects: linux-nvidia-bos (Ubuntu Noble)
Importance: Undecided
Status: New
** Description changed:
+ SRU Justification
+
+ [Impact]
+
+ There is a performance regression seen for calls to get_user_pages() on
+ single-page NULL-mapping folios, introduced by commit f002882ca369 ("mm: merge
+ folio_is_secretmem() and folio_fast_pin_allowed() into
+ gup_fast_folio_allowed()").
+
+ When either CONFIG_SECRETMEM=y, or when the folio is a long-term writable pin,
+ the current implementation requires that the mapping be checked as either a
+ secretmem mapping or a file-backed one. If it is either, or if the mapping is
+ NULL, the slow path is forced.
+
+ However, when the mapping is NULL, the folio cannot be a secretmem mapping.
+ Thus, as long as the folio is NOT a long-term writable pin, the fast path can
+ still be used.
+
+ This performance regression was initially observed during GPU Direct Storage
+ (GDS) workloads.
+
+ [Fix]
+
+ The upstream commit c494788faffe ("mm/gup: fix GUP-fast fallback for
+ NULL-mapping order-0 folios") resolves the performance regression by changing
+ gup_fast_folio_allowed() to allow the fast path for the case described above,
+ where the folio is NOT a long-term writable pin and its mapping field is NULL.
+
+ [Test Plan]
+
+ Build and boot tested.
+
+ The performance regression and subsequent fix can be verified with a GDS
+ workload. The original bug report also describes a test kernel module that
uses
+ the `alloc_page` + `vm_insert_page` + `pin_user_pages_fast(..., FOLL_WRITE,
+ ...)` functions to emulate GDS behavior, and using
`get_user_pages_fast_only()`
+ on that to validate that the fast path is now allowed in this scenario.
+
+ [Where problems could occur]
+
+ The fix affects the get_user_pages*() path of the mm subsystem, which is used
+ heavily. The fix has been reviewed and applied in upstream Linux.
+
+ -------------------------------------------------------
+
+ Original report:
+
Clean cherry-pick from linux-next:
```
(cherry picked from commit ae75e88d8c258fd849de594e7d468b5263e7b3e3
linux-next)
```
John Hubbard, Acked-by David Hildenbrand, applied by Andrew Morton.
[Lore
thread](https://lore.kernel.org/all/[email protected]/).
## Problem
`f002882ca369` (present on this branch) made `gup_fast_folio_allowed()` bail
to the
slow path for *any* order-0 folio with a NULL `->mapping` when
`CONFIG_SECRETMEM=y`.
Pages from `alloc_page()` + `vm_insert_page()` legitimately have a NULL
mapping, so
every `pin_user_pages_fast()` over such a range misses the fast path —
nvidia-fs
(GPUDirect Storage) allocates its shadow buffers exactly this way.
The NULL check was meant to catch truncated file-backed pages, not secretmem.
Secretmem folios are published via `filemap_add_folio()`, which always sets
`->mapping`, so a NULL mapping proves the folio is *not* secretmem. Returning
`!reject_file_backed` keeps long-term writable pins on the slow path and
restores
the fast path otherwise — exactly the pre-`f002882ca369` behaviour.
`CONFIG_SECRETMEM=y` on amd64 and arm64, so this is live on every
flavour we ship.
## Measurement (GH200, 288 cores)
A module reproducing the nvidia-fs pattern (`alloc_page` + `vm_insert_page`,
then
`pin_user_pages_fast(..., FOLL_WRITE, ...)`), with an anonymous-memory
control the
patch cannot affect. `get_user_pages_fast_only()` gives the GUP-fast verdict
directly: **0/N unpatched, N/N patched**.
Median ns/page, 512 pages/thread, 5 reps:
| threads | 4K unpatched → patched | 64K unpatched → patched |
|--------:|-----------------------:|------------------------:|
| 1 | 34 → 34 (1.0x) | 33 → 30 (1.1x) |
| 8 | 74 → 33 (**2.2x**) | 1920 → 31 (**62x**) |
| 32 | 95 → 33 (**2.9x**) | 4792 → 33 (**145x**) |
| 128 | 500 → 218 (noisy) | 16673 → 76 (**219x**) |
The anon control held at 30-35 ns/page across all four kernels, so only the
affected
range moved. Stock `7.0.0-1015-nvidia-64k` independently reproduces the
unpatched 64K
numbers (8 threads: 1176 vs 37), so this isn't a test-config artefact.
Single-threaded it's a wash; the win is under concurrency. `perf` on the
unpatched
64K kernel shows the slow path is ~90% lock contention
(`queued_spin_lock_slowpath`
72%), gone entirely once patched — contention that scales with thread count,
not a
fixed per-page cost.
## Risk
Low. One line in a static function with three callers, all in GUP-fast. Only
`reject_file_backed == false && check_secretmem && mapping == NULL` changes
behaviour; long-term writable pins are untouched.
A secretmem folio caught mid-truncate can't happen here: `secretmem_setattr()`
refuses to shrink, there's no `.fallocate` (so no punch-hole), and
`secretmem_migrate_folio()` returns `-EBUSY`. Only inode eviction remains,
which
requires every VMA gone — no VMA, no PTE for GUP-fast to walk. Raised by David
Hildenbrand on v1 and resolved before he Acked.
## Testing
Applies cleanly; built arm64 4K/64K and x86_64, no new warnings; verified in
the
binary (unpatched `mov w0, #0x0` vs patched `eor w0, w0, #0x1`); benchmarked
as above.
** Changed in: linux (Ubuntu Noble)
Status: New => In Progress
** Changed in: linux (Ubuntu Noble)
Assignee: (unassigned) => Jacob Martin (jacobmartin)
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2162917
Title:
Backport: "mm/gup: fix GUP-fast fallback for NULL-mapping order-0
folios"
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2162917/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
