On Fri, Apr 24, 2026 at 4:10 PM Ihor Solodrai <[email protected]> wrote: > > I wonder if it's feasible to implement KASAN support on the verifier > side in post-verification fixups. AI slop for illustration: > > ;; Original (1 BPF insn): > dst = *(u64 *)(src + off) ; BPF_LDX | BPF_MEM | BPF_DW > > ;; Rewrite (~7 BPF insns): > r_tmp1 = src ; BPF_MOV64_REG > r_tmp1 += off ; BPF_ALU64 | BPF_ADD | K (full > address) > r_tmp2 = r_tmp1 ; copy > r_tmp2 >>= 3 ; KASAN_SHADOW_SCALE_SHIFT > r_tmp2 += KASAN_SHADOW_OFFSET ; shadow address > r_tmp3 = *(u8 *)(r_tmp2 + 0) ; BPF_LDX | BPF_B (load shadow byte) > if r_tmp3 != 0 goto +2 ; BPF_JNE | PC+2 > dst = *(u64 *)(src + off) ; original access (fast path) > goto +1 ; skip slowpath > call __asan_report_load8 ; BPF kfunc > dst = *(u64 *)(src + off) ; retry the access after report > (non-fatal) > > A sort of inline kasan directly in BPF. > > There are plenty of issues with it: instruction limit, exposing asan > API as kfuncs, etc. On the flip side we get cross-arch support out of > the box with no or mininal JIT changes. > > Honestly I'm not excited about this approach, but curious if anyone > thought about this, or maybe it was already discussed?
We discussed this. It won't work because we don't have that many temp registers for once and second it has to preserve all (both callee and caller saved regs). This is arch specific. Second, we do not want other archs. This feature is x86-64 only. It's being added to find _verifier_ bugs. To do that one arch is enough. > > > - not all memory accessing BPF instructions are being instrumented: > > - it focuses on STX/LDX instructions > > - it discards instructions accessing BPF program stack (already > > monitored by page guards) > > - it discards possibly faulting instructions, like BPF_PROBE_MEM or > > BPF_PROBE_ATOMIC insns > > > > The series is marked and sent as RFC: > > - to allow collecting feedback early and make sure that it goes into the > > right direction > > - because it depends on Xu's work to pass data between the verifier and > > JIT compilers. This work is not merged yet, see [2]. I have been > > tracking the various revisions he sent on the ML and based my local > > branch on his work > > - because tests brought by this series currently can't run on BPF CI: > > they expect kasan multishot to be enabled, otherwise the first test > > will make all other kasan-related tests fail. > > AFAICT this can be trivially fixed on BPF CI side, we just need to set > kasan_multi_shot for the VMs running the tests. I will do that, your > next revision doesn't have to be and RFC. +1 > > - because some cases like atomic loads/stores are not instrumented yet > > (and are still making me scratch my head) > > - because it will hopefully provide a good basis to discuss the topic at > > LSFMMBPF (see [3]) > > Apparently, KASAN reporting routine takes a lock [1]: > > __asan_load() > -> check_region_inline() > -> kasan_report() > -> start_report() > -> raw_spin_lock_irqsave(&report_lock, *flags); > > BPF programs can run in NMI context, and so it appears to be possible > to get an unflagged (because of lockdep_off() in start_report) > deadlock, if an NMI fires on a CPU already holding report_lock. > Although I guess you'd need two KASAN bugs to happen > simultaneously for that to occur?... A rare event, I would hope. > > It could be addressed with either in_nmi() check at runtime, or > forbidding kasan for NMI-runnable BPF program types. We don't need that. If this bpf KASAN finds a bug, it means that it found a verifier bug. All things are out of the window. kasan_report() splat can just as well be the last thing that users will see.
