On Tue, Jul 21, 2026 at 4:18 AM Mina Almasry <[email protected]> wrote: >
Hi Mina, Thanks a lot for your review! > On Sun, Jul 19, 2026 at 11:01 AM Taehee Yoo <[email protected]> wrote: > > > > knod (in-kernel network offload device) drives a GPU directly from the > > Linux kernel - with no userspace GPU runtime such as CUDA or ROCm, and > > no userspace component in the data path - to accelerate packet > > processing. > > > > The kernel itself allocates the GPU's queues, JIT-compiles the > > per-packet program to GPU machine code, and dispatches it; the NIC DMAs > > received packets straight into GPU memory and the GPU returns a verdict. > > The GPU is programmed like any other in-kernel offload, not through a > > userspace framework, so existing XDP programs and IPsec SAs can be > > offloaded to it transparently. > > > > The design and motivation were presented at Linux Plumbers Conference > > 2025: > > > > https://lpc.events/event/19/contributions/2267/ > > > > Motivation > > ========== > > > > Line-rate packet processing that does non-trivial per-packet work - an > > XDP program doing L4 load balancing, or IPsec crypto - is bound by the > > host CPU: each core handles one packet at a time, so scaling means > > spending more cores. A GPU is the opposite shape - thousands of lanes > > running the same small program over many packets at once (SIMT) - which > > happens to match the per-packet-program model of XDP. > > > > knod moves that per-packet compute off the host CPU and onto a GPU. The > > NIC DMAs received packets directly into GPU memory, the GPU runs the > > program across a batch of packets in parallel, and only the result > > comes back: a verdict for every packet, plus the packet itself for the > > ones destined to the host. The CPU no longer pays the per-packet > > program cost, and throughput scales with GPU occupancy rather than core > > count. > > > > Crucially this happens entirely inside the kernel. GPU packet processing > > today generally launches work from a userspace GPU runtime (CUDA and > > friends) and keeps that runtime in the data path; knod instead builds > > the GPU queues, compiles the program, and dispatches it from the kernel, > > so it plugs into existing offload paths (XDP, xfrm) with nothing to > > install or keep running in userspace. > > > > TBH I found this motivation weak, especially since IIUC you're asking > for almost ~50K lines of code to be merged to the kernel. My thinking > is that (a) it's true that Native XDP takes up CPU cores, but > HW-offload XDP already exists and takes up no CPU. (b) with AI, GPUs > are very expensive and the work they're doing is critical, so you're > unlikely to buy a GPU and use it for knod-offloaded XDP; you'd > probably buy a cheaper smart-NIC? And in the cases where you do have a > GPU, it's likely your system's money-maker, and you probably want to > offload work from the GPU, not to your GPU. > Thanks, these are fair concerns. For a system where the GPU is already fully utilized, a SmartNIC may indeed be the better target for XDP offload. We are not arguing that GPU offload is always cheaper or more efficient. The goal of KNOD is to provide an upstream path for using programmable accelerators already available in the system for kernel networking. Hardware XDP offload exists, but the available functionality is tied to the NIC, its firmware, and the features maintained by the vendor. KNOD instead aims to provide an execution layer that can be extended in the kernel community. XDP is the first workload used to validate this layer, rather than the intended limit of KNOD. > But as far as I can tell there should be much more interesting > applications for what you're doing rather than offloading XDP. Like > wouldn't you with this feature be able to implement ML collectively > like all-to-all/all-reduce/all-gather as purely XDP programs offloaded > to the GPU? If you have that implemented and can positively compare > performance to RoCE or RDMA that would be a much more interesting use > case IMO. > I agree that this is a promising direction, and collective communication could be a good fit for this model. At least for all-reduce, however, BPF/XDP does not currently support the required floating-point operations. If that support is discussed and added by the BPF community, this would be an interesting use case to explore with KNOD and evaluate against RoCE or RDMA. > > Model > > ===== > > > > knod binds two endpoints through a third object: > > > > - a NIC-side netdev, registered by the NIC driver; > > - an accelerator (the GPU), registered by a provider built into the > > GPU driver; > > - an offload device, created on attach, that connects them and owns > > the per-queue data-path state. > > > > The accelerator ops are feature-agnostic, so different per-packet > > programs plug in behind one interface. This series ships two features > > to show the framework is not tied to a single use case (the accel-type > > uAPI also reserves "dpu" for future non-GPU backends): > > > > - BPF: an XDP program attached in offload mode is JIT-compiled from > > eBPF to an AMD GCN shader and executed on the GPU. > > - IPsec: RX ESP full-packet decrypt on the GPU (proof of concept, > > see below). > > > > Data path > > ========= > > > > For a NIC RX queue bound to an accelerator, a received packet flows as > > follows: > > > > 1. Attach reconfigures the NIC's page_pool so its pages come from GPU > > memory, exported as a dma-buf and plugged in through the devmem > > memory provider. The NIC therefore DMAs the packet straight into > > GPU memory - there is no host-side copy on RX. > > > > 2. Instead of building an skb and entering the stack, the NIC's NAPI > > pushes a small descriptor (memory ref + offset + length) onto a > > per-queue lock-free SPSC ring shared with the GPU. > > > > 3. A persistent GPU worker drains descriptors and runs the active > > feature's shader over a batch of packets in parallel - one > > workgroup per packet - then writes a per-packet verdict back onto > > the ring: PASS, DROP or TX. > > > > 4. Back on the NIC NAPI, each verdict is applied: > > - DROP: the GPU page is recycled to the pool; > > - TX: the packet is sent back out the NIC directly from GPU > > memory (XDP_TX), with no host round-trip; > > - PASS: the packet is copied out of GPU memory into a host > > delivery page by the GPU's DMA engine (async SDMA), > > wrapped as an skb, and handed to the normal stack. > > > > So only PASS packets ever touch host memory, and even then the copy is > > done by the GPU, not the CPU. > > > > The control plane is a generic netlink family ("knod") for > > attach/detach, accelerator/NIC inventory, and per-accelerator feature > > selection, with notifications on bind/unbind. > > > > Code layout > > =========== > > > > knod separates into a subsystem-neutral core and a GPU-specific > > provider; nothing in the core knows about GPUs. Roughly: > > > > - core + control plane + NIC drivers net/, kernel/bpf/ (~4.4k) > > - GPU provider (queues, JIT, shaders) drivers/gpu/drm/amd/ (~39k) > > > > Since the changes are so large, I would break this work into many > series, the first which only supports a very minimal set of work > (which I would guess would be XDP_DROP and XDP_TX), and then follow up > with another series which adds XDP_PASS support, and then IPSec, etc. > That makes sense. We will split the work and start with the minimum functionality needed to demonstrate the architecture, likely XDP_DROP and XDP_TX. XDP_PASS, IPsec, and the remaining functionality can follow in separate series. Thanks a lot! Taehee Yoo > -- > Thanks, > Mina
