This patch adds BPF verifier support for multi-level pointer parameters
and return values in BPF trampolines. The implementation treats these
parameters as PTR_TO_MEM with read-only semantics, applying either
untrusted or trusted access patterns while honoring __nullable
annotations. Runtime safety is ensured through existing exception
handling mechanisms for untrusted memory reads, with the verifier
enforcing bounds checking and null validation. The series includes
selftests covering double and triple pointer arguments across
fentry/fexit/lsm programs and verifier context validation.
Background:
Prior to these changes, accessing multi-level pointer parameters or
return values through BPF trampoline context arrays resulted in
verification failures in btf_ctx_access, producing errors such as:
func '%s' arg%d type %s is not a struct
For example, consider a BPF program that logs an input parameter of type
struct posix_acl **:
SEC("fentry/__posix_acl_chmod")
int BPF_PROG(trace_posix_acl_chmod, struct posix_acl **ppacl, gfp_t gfp,
umode_t mode)
{
bpf_printk("__posix_acl_chmod ppacl = %px\n", ppacl);
return 0;
}
This program failed BPF verification with the following error:
libbpf: prog 'trace_posix_acl_chmod': -- BEGIN PROG LOAD LOG --
0: R1=ctx() R10=fp0
; int BPF_PROG(trace_posix_acl_chmod, struct posix_acl **ppacl,
gfp_t gfp, umode_t mode) @ posix_acl_monitor.bpf.c:23
0: (79) r6 = *(u64 *)(r1 +16) ; R1=ctx() R6_w=scalar()
1: (79) r1 = *(u64 *)(r1 +0)
func '__posix_acl_chmod' arg0 type PTR is not a struct
invalid bpf_context access off=0 size=8
processed 2 insns (limit 1000000) max_states_per_insn 0 total_states 0
peak_states 0 mark_read 0
-- END PROG LOAD LOG --
The common workaround involved using helper functions to fetch parameter
values by passing the address of the context array entry:
SEC("fentry/__posix_acl_chmod")
int BPF_PROG(trace_posix_acl_chmod, struct posix_acl **ppacl, gfp_t gfp,
umode_t mode)
{
struct posix_acl **p;
bpf_probe_read_kernel(&p, sizeof(ppacl), &ctx[0]);
bpf_printk("__posix_acl_chmod before %px\n", p);
return 0;
}
This approach introduced helper call overhead and created inconsistency
with parameter access patterns.
Improvements:
With this patch, trampoline programs can directly read parameters and
dereference memory using load instructions, eliminating helper call
overhead and ensuring consistent parameter handling. For example, the
following helper call sequence:
{
struct posix_acl **pp;
struct posix_acl *p;
bpf_probe_read_kernel(&pp, sizeof(pp), &ctx[0]);
bpf_probe_read_kernel(&p, sizeof(p), pp);
...
}
can be replaced by two load instructions implementing a single C
statement:
{
struct posix_acl *p = *ppacl;
...
}
Design Rationale: PTR_TO_MEM vs SCALAR
The verifier assigns SCALAR type to single-level pointers (void*, int*).
For multi-level pointers, I selected PTR_TO_MEM to enable memory access
through a single load instruction for the first level of dereference,
with subsequent dereferences becoming SCALAR. This design eliminates
helper call for parameter dereference, replacing it with a load
instruction (e.g., void* ptr = *pptr).
Access safety is maintained through existing verify-time checks,
exception handling, and kernel virtual address range boundary checks:
- User-mode memory address access is prevented by runtime virtual
address range checks for untrusted PTR_TO_MEM
- Invalid kernel address space accesses are intercepted by the
exception handler for untrusted PTR_TO_MEM
- Trusted PTR_TO_MEM access safety is maintained at verify time
v1 -> v2: corrected maintainer's email
Slava Imameev (2):
bpf: Support multi-level pointer params via PTR_TO_MEM for trampolines
selftests/bpf: Add trampolines multi-level pointer params test
coverage
include/linux/bpf.h | 3 +-
kernel/bpf/btf.c | 54 ++-
kernel/bpf/verifier.c | 4 +-
net/bpf/test_run.c | 128 ++++++
.../prog_tests/fentry_fexit_multi_level_ptr.c | 204 +++++++++
.../selftests/bpf/prog_tests/verifier.c | 2 +
.../progs/fentry_fexit_pptr_nullable_test.c | 52 +++
.../bpf/progs/fentry_fexit_pptr_test.c | 60 +++
.../bpf/progs/fentry_fexit_void_ppptr_test.c | 31 ++
.../bpf/progs/fentry_fexit_void_pptr_test.c | 64 +++
.../bpf/progs/verifier_ctx_multilevel_ptr.c | 429 ++++++++++++++++++
11 files changed, 1021 insertions(+), 10 deletions(-)
create mode 100644
tools/testing/selftests/bpf/prog_tests/fentry_fexit_multi_level_ptr.c
create mode 100644
tools/testing/selftests/bpf/progs/fentry_fexit_pptr_nullable_test.c
create mode 100644 tools/testing/selftests/bpf/progs/fentry_fexit_pptr_test.c
create mode 100644
tools/testing/selftests/bpf/progs/fentry_fexit_void_ppptr_test.c
create mode 100644
tools/testing/selftests/bpf/progs/fentry_fexit_void_pptr_test.c
create mode 100644
tools/testing/selftests/bpf/progs/verifier_ctx_multilevel_ptr.c
--
2.50.1 (Apple Git-155)
