On Sun, Aug 16, 2026 at 11:45:24PM +0100, Kiryl Shutsemau wrote:
>From: "Kiryl Shutsemau (Meta)" <[email protected]>
>
>Fill in the allocation, which happens on both sides of the freeze.
>
>A destination is a folio of the candidate's order, charged to the memcg,
>with the memcg's deferred-split list entry taken up front while sleeping
>is still allowed: the PMD-order install would otherwise need one under
>the pmd lock.
>
>collapse_alloc() does all of that for one candidate with the gfp it is
>handed, and counts nothing when it fails: what a miss means is up to the
>caller.
>
>collapse_provision() is the caller inside the window. The sources are
>frozen by then and a faulter on any of them is waiting, so it asks
>without __GFP_DIRECT_RECLAIM: reclaim entered there would be paid for by
>that faulter.
[...]
>+/*
>+ * Allocate one candidate's destination with @gfp: a folio of its order,
>charged,
>+ * with the memcg's deferred-split list heads in place so the install cannot
>need
>+ * to allocate under the pmd lock. Those heads cost only the first collapse
>in a
>+ * memcg.
>+ *
>+ * A failure counts nothing and changes nothing: what a miss means is the
>caller's
>+ * policy.
>+ */
>+static enum scan_result collapse_alloc(struct mm_struct *mm,
>+ struct collapse_control *cc,
>+ struct collapse_candidate *cand,
>+ gfp_t gfp)
>+{
>+ struct folio *folio;
>+
>+ folio = __folio_alloc(gfp, cand->order, collapse_find_target_node(cc),
>+ &cc->alloc_nmask);
>+ if (!folio)
>+ return SCAN_ALLOC_HUGE_PAGE_FAIL;
>+
>+ if (unlikely(mem_cgroup_charge(folio, mm, gfp)) ||
>+ folio_memcg_alloc_deferred(folio)) {
One small nit: folio_memcg_alloc_deferred() passes GFP_KERNEL to the
deferred_split_lru allocation.
At the full series tip, collapse_round() invokes it after
collapse_freeze() and before collapse_putback(), while faults on the
source migration entries wait:
static void collapse_round(struct mm_struct *mm, unsigned long pmd_addr,
struct collapse_control *cc)
{
...
collapse_freeze(vma, cc, pmd);
collapse_provision(mm, cc);
...
collapse_putback(vma, cc);
...
}
collapse_provision() strips direct reclaim before calling
collapse_alloc():
static void collapse_provision(struct mm_struct *mm,
struct collapse_control *cc)
{
const gfp_t gfp = cc->policy.gfp & ~__GFP_DIRECT_RECLAIM;
...
result = collapse_alloc(mm, cc, cand, gfp);
...
}
collapse_alloc() uses that gfp for the charge, then calls
folio_memcg_alloc_deferred():
static enum scan_result collapse_alloc(struct mm_struct *mm,
struct collapse_control *cc,
struct collapse_candidate *cand,
gfp_t gfp)
{
...
if (unlikely(mem_cgroup_charge(folio, mm, gfp)) ||
folio_memcg_alloc_deferred(folio)) {
...
}
...
}
The helper substitutes GFP_KERNEL:
int folio_memcg_alloc_deferred(struct folio *folio)
{
...
return folio_memcg_list_lru_alloc(folio, &deferred_split_lru,
GFP_KERNEL);
}
When this metadata has not been allocated for a memcg yet,
folio_memcg_list_lru_alloc() can take its blocking slow branch:
int folio_memcg_list_lru_alloc(struct folio *folio, struct list_lru *lru,
gfp_t gfp)
{
...
/* Fast path when list_lru heads already exist */
rcu_read_lock();
memcg = folio_memcg(folio);
res = memcg_list_lru_allocated(memcg, lru);
rcu_read_unlock();
if (likely(res))
return 0;
/* Allocation may block, pin the memcg */
memcg = get_mem_cgroup_from_folio(folio);
res = __memcg_list_lru_alloc(memcg, lru, gfp);
...
}
GFP_KERNEL includes __GFP_DIRECT_RECLAIM:
...
#define __GFP_DIRECT_RECLAIM ((__force gfp_t)___GFP_DIRECT_RECLAIM) /*
Caller can reclaim */
#define __GFP_KSWAPD_RECLAIM ((__force gfp_t)___GFP_KSWAPD_RECLAIM) /*
kswapd can wake */
#define __GFP_RECLAIM ((__force
gfp_t)(___GFP_DIRECT_RECLAIM|___GFP_KSWAPD_RECLAIM))
...
#define GFP_KERNEL (__GFP_RECLAIM | __GFP_IO | __GFP_FS)
This only affects calls that find the metadata missing; after it has been
installed, later calls take the fast branch. Still, it bypasses the gfp
with direct reclaim stripped for the freeze window. Could
folio_memcg_alloc_deferred() take that gfp here?
[...]
Cheers, Lance
