On Thu, Feb 26, 2026 at 02:27:24PM +1100, Alistair Popple wrote:
> On 2026-02-25 at 02:17 +1100, Gregory Price <[email protected]> wrote...
> >
> > If your service only allocates movable pages - your ZONE_NORMAL is
> > effectively ZONE_MOVABLE.
>
> This is interesting - it sounds like the conclusion of this is ZONE_* is just
> a
> bad abstraction and should be replaced with something else maybe some like
> this?
>
> And FWIW I'm not tied to the ZONE_DEVICE as being a good abstraction, it's
> just
> what we seem to have today for determing page types. It almost sounds like
> what
> we want is just a bunch of hooks that can be associated with a range of pages,
> and then you just get rid of ZONE_DEVICE and instead install hooks appropriate
> for each page a driver manages. I have to think more about that though, this
> is just what popped into my head when you start saying ZONE_MOVABLE could also
> disappear :-)
>
... snip ...
> >
> > You don't have to squint because it was deliberate :]
>
> Nice.
>
I've had some time to chew on this a bit more.
Adding a node-scope `struct dev_pagemap` produces some interesting
(arguably useful / valuable) effects.
The invariant would be clamping the entire node to ZONE_DEVICE
(more on this below).
So if we think about it this way - we could just view this whole thing
as another variant of ZONE_DEVICE - but without needing the memremap
infrastructure (you can use normal hotplug to achieve it).
0. pgdat->private becomes pgdat->dev_pagemap
N_MEMORY_PRIVATE -> N_MEMORY_DEVICE ?
As a start, do a direct conversion, and use the existing
infrastructure. then expand hooks as needed (and as-is reasonable)
Some of the `struct dev_pagemap {}` fields become dead at the node
scope, but this is a plumbing issue.
There's already an similar split between the dev_pagemap and the ops
structure, so it might map very cleanly.
1. "Clamping the entire node to ZONE_DEVICE"
When we do this, the *actual* ZONE becomes completely irrelevant.
The allocation path is entirely controlled, so you might actually end
up freeing up the folio flags that track the zone:
static inline enum zone_type memdesc_zonenum(memdesc_flags_t flags)
{
ASSERT_EXCLUSIVE_BITS(flags.f, ZONES_MASK << ZONES_PGSHIFT);
return (flags.f >> ZONES_PGSHIFT) & ZONES_MASK;
}
becomes:
folio_is_zone_device(folio) {
return node_is_device_node(folio_nid(folio)) ||
memdesc_is_zone_device(folio->flags);
}
Kind of an interesting. You still need these flags for traditional
ZONE_DEVICE, so you can't evict it completely, but you can start to
see a path here.
2. One dev_pagemap per node or multiple w/ pagemap range searching
Checking membership is always cheap:
node_is_device_node()
Getting ops can be cheap if 1:1 mappings exists:
pgdat->device_ops->callback()
Or may be expensive if range-based matching is required:
node_device_op(folio, ...) {
ops = node_ops_lookup(folio); /* pfn-range binary search */
ops->callback(folio, ...)
}
pgmap already has an embedded range:
struct dev_pagemap {
...
int nr_range;
union {
struct range range;
DECLARE_FLEX_ARRAY(struct range, ranges);
};
};
Example: Nouveau, registers hundreds of pgmap instances that it
uses to recover driver contexts for that specific folio.
This would not scale well.
But most other drivers register between 1-8. That might.
That means this might actually be an effective way to evict pgmap
from struct folio / struct page. (Not making this a requirement or
saying it's reasonable, just an interesting observation).
3. Some existing drivers with 1 pgmap per driver instance instantly get
the folio->lru field back - even if they continue to use ZONE_DEVICE.
At least 3 drivers use page->zone_device_data as a page freelist
rather than actual per-page data. Those drivers could just start
using folio/page->lru instead.
Some store actual per-page zone_device_data that would prevent this,
but from poking around it seems like it might be feasible.
Some use the pgmap as a container_of() argument to get driver
context, may or may not be supportable out of the box, but it seemed
like mild refactoring might get them back the use of folio->lru.
None of this is required, the goal is explicitly not disrupting any
current users of ZONE_DEVICE.
Just some additional food for thought.
As-designed now, this would only apply to NUMA systems, meaning you
can't fully evict pgmap from struct page/folio --- but you could
imagine a world where in non-numa mode we even register a separate
pglist_data specifically for device memory even w/o NUMA.
~Gregory
