During early boot (early_trace_init() called from start_kernel()),
tracing allocates initial ring buffers (temp_buffer, array_buffer, and
snapshot_buffer) for CPU 0. Later, as secondary CPUs come online during
smp_init(), each CPU's ring buffer is allocated via trace_rb_cpu_prepare().
Before attempting page allocation, __rb_allocate_pages() performs a
heuristic check using si_mem_available() to return early with -ENOMEM if
memory appears insufficient.
However, on kernels built with CONFIG_DEFERRED_STRUCT_PAGE_INIT=y,
NR_FREE_PAGES only reflects the initial non-deferred pool across the entire
period from early_trace_init() through smp_init(). defer_init() leaves
only a single section per node (e.g. 16 MiB with 64 KB pages) initialized
up-front, and on kernels with large static binary footprints (debug
configurations enabling PAGE_OWNER, DEBUG_PAGEALLOC, KFENCE, or SLUB_DEBUG),
early core initialisations (SLUB caches, vmalloc, static ftrace records)
consume virtually all managed pages in this pool.
si_mem_available() therefore returns a false negative (often 0-1 pages)
across both SYSTEM_BOOTING (for the initial CPU 0 buffers) and
SYSTEM_SCHEDULING (for secondary CPU buffers via trace_rb_cpu_prepare()),
causing __rb_allocate_pages() to prematurely abort with -ENOMEM.
This failure is false: the buddy allocator is designed to replenish
ZONE_NORMAL on demand via deferred_grow_zone() when an actual allocation
is attempted. Checking si_mem_available() before calling alloc_pages_node()
short-circuits this on-demand growth entirely.
deferred_grow_zone() is only permanently disabled after page_alloc_init_late()
completes (it spawns per-node kthreads and blocks until done), which happens
inside kernel_init_freeable() — before system_state is set to
SYSTEM_FREEING_INITMEM. Skip the si_mem_available() check until
SYSTEM_FREEING_INITMEM, at which point all deferred pages are initialized
and si_mem_available() is accurate.
Wrap the guard in #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT so that on kernels
without deferred init the original check is preserved unchanged.
Fixes: 2a872fa4e9c8 ("ring-buffer: Check if memory is available before
allocation")
Cc: [email protected]
Reported-by: Michal Suchánek <[email protected]>
Closes: https://lore.kernel.org/all/[email protected]/
Link: https://lore.kernel.org/all/[email protected]/
Signed-off-by: Amit Machhiwal <[email protected]>
---
Changes since v2:
- Rebase on top of upstream ring_buffer.c changes from Vincent Donnefort;
update threshold to i < (nr_pages << subbuf_order). (Michal Suchánek)
Changes since v1:
- Extend the guard to cover SYSTEM_SCHEDULING as well: smp_init() calls
trace_rb_cpu_prepare() -> __rb_allocate_pages() before page_alloc_init_late()
completes, so the false negative window spans both SYSTEM_BOOTING and
SYSTEM_SCHEDULING. Use system_state >= SYSTEM_FREEING_INITMEM as the correct
boundary. (Sashiko, Vincent Donnefort)
- Wrap in #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT to preserve the original
check unchanged on non-deferred kernels. (Vincent Donnefort)
- Drop Tested-by: guard condition changed, requesting fresh test.
kernel/trace/ring_buffer.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c
index 9c4e40847e43..8e5a59a7b69b 100644
--- a/kernel/trace/ring_buffer.c
+++ b/kernel/trace/ring_buffer.c
@@ -2463,10 +2463,27 @@ static int __rb_allocate_pages(struct
ring_buffer_per_cpu *cpu_buffer,
* memory. It may not be accurate. But we don't care, we just
want
* to prevent doing any allocation when it is obvious that it is
* not going to succeed.
+ *
+ * With CONFIG_DEFERRED_STRUCT_PAGE_INIT, NR_FREE_PAGES only
+ * reflects the initial non-deferred pool during early boot and
+ * throughout SMP bring-up (smp_init() runs before
+ * page_alloc_init_late() completes deferred initialization).
+ * si_mem_available() returns a false negative across this
+ * entire window while actual allocations succeed by growing the
+ * zone on demand via deferred_grow_zone(). Only re-enable the
+ * check from SYSTEM_FREEING_INITMEM onward, after
+ * page_alloc_init_late() has finished and si_mem_available() is
+ * accurate.
*/
i = si_mem_available();
+#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
+ if (system_state >= SYSTEM_FREEING_INITMEM &&
+ i < (nr_pages << subbuf_order))
+ return -ENOMEM;
+#else
if (i < (nr_pages << subbuf_order))
return -ENOMEM;
+#endif
user_thread = current->mm != NULL;
}
base-commit: 7be4bc52fd9e0e55dc9a2e0232b89427d2c4b614
--
2.54.0 (Apple Git-157)
