On Tue, May 20, 2025 at 06:42:11PM +0200, David Hildenbrand wrote: > A long-term goal is supporting frozen PageOffline pages, and later > PageOffline pages that don't have a refcount at all. Some more work for > that is needed -- in particular around non-folio page migration and > memory ballooning drivers -- but let's start by handling PageOffline pages > that can be skipped during memory offlining differently. > > Note that PageOffline is used to mark pages that are logically offline > in an otherwise online memory block (e.g., 128 MiB). If a memory > block is offline, the memmap is considered compeltely uninitialized > and stale (see pfn_to_online_page()). > > Let's introduce a PageOffline specific page flag (PG_offline_skippable) > that for now reuses PG_owner_2. In the memdesc future, it will be one of > a small number of per-memdesc flags stored alongside the type. > > By setting PG_offline_skippable, a driver indicates that it can > restore the PageOffline state of these specific pages when re-onlining a > memory block: it knows that these pages are supposed to be PageOffline() > without the information in the vmemmap, so it can filter them out and > not expose them to the buddy -> they stay PageOffline(). > > While PG_offline_offlineable might be clearer, it is also super > confusing. Alternatives (PG_offline_sticky?) also don't quite feel right. > So let's use "skippable" for now. > > The flag is not supposed to be used for movable PageOffline pages as > used for balloon compaction; movable PageOffline() pages can simply be > migrated during the memory offlining stage, turning the migration > destination page PageOffline() and turning the migration source page > into a free buddy page. > > Let's convert the single user from our MEM_GOING_OFFLINE approach > to the new PG_offline_skippable approach: virtio-mem. Fortunately, > this simplifies the code quite a lot. The only corner case we have to > take care of is when force-unloading the virtio-mem driver: we have to > prevent partially-plugged memory blocks from getting offlined by > clearing PG_offline_skippable again. > > What if someone decides to grab a reference on these pages although they > really shouldn't? After all, we'll now keep the refcount at 1 (until we > can properly stop using the refcount completely). > > Well, less worse things will happen than would currently: currently, > if someone would grab a reference to these pages, in MEM_GOING_OFFLINE > we would run into the > if (WARN_ON(!page_ref_dec_and_test(page))) > dump_page(page, "fake-offline page referenced"); > > And once that unexpected reference would get dropped, we would end up > freeing that page to the buddy: ouch. > > Now, we'll allow for offlining that memory, and when that unexpected > reference would get dropped, we would not end up freeing that page to > the buddy. Once we have frozen PageOffline() pages, it will all get a > lot cleaner. > > Note that we didn't see the existing WARN_ON so far, because nobody > should ever be referencing such pages. > > An alternative might be to have another callback chain from memory hotplug > code, where a driver that owns that page could agree to skip the > PageOffline() page. However, we would have to repeatedly issue these > callbacks for individual PageOffline() pages, which does not sound > compelling. As we have spare bits, let's use this simpler approach for > now. > > Acked-by: Zi Yan <z...@nvidia.com> > Signed-off-by: David Hildenbrand <da...@redhat.com>
Acked-by: Oscar Salvador <osalva...@suse.de> -- Oscar Salvador SUSE Labs
