Hiho, Thx for the nice summary! From my more admin, testing, running Mesh network perspective on OpenWrt I would prefer option 4)
Reasons: -eBPF filters running in OpenWrt should be a reviewed process overseen by the core team -the CO-RE concept with userspace scripting for mature eBPF filters sounds like that it could be made solid -eBPF filter development vs. its usage of a robus subset of eBPFs… I prefer the latter one Greetings Thomas > On 3. Oct 2021, at 14:44, Felix Fietkau <n...@nbd.name> wrote: > > > Hi, > > I recently spent some time digging into what's needed for proper eBPF > build support in OpenWrt. Here's what I found so far: > > Most out-of-tree eBPF based projects fork some of the BPF related kernel > headers from various different kernel versions and manually maintain > those forks. These header files are usually very incomplete and tailored > specifically for the project that uses them. To make things even worse, > they typically explicitly rely on including headers from the local x86 > host toolchain header files when building for the BPF target. > > The in-kernel build of BPF modules is weird in a different way. It > explicitly includes all the header files from the arch that the kernel > is being built for. And because a lot of assembly stuff in there is > completely incompatible with building for the BPF target, the build > system actually targets clang to the same arch that the kernel is being > built for and only emits un-optimized LLVM bitcode without running any > of the LLVM passes. It then passes that bitcode to the optimizer and > target code generator with bpf specified as target arch. > It's a weird hack, but it seems to work properly even when > cross-compiling to non-x86 targets. > > When writing ebpf modules myself, I definitely don't want to rely on the > crappy header fork mess that most out-of-tree projects use. In my tests > it was simply too fragile, and I couldn't get it to work on my macOS > build host either. Even if we could make it work for more use cases, it > would still be a maintenance hell when comes to supporting more features > and newer kernel versions. > > I think staying close to the way that in-tree BPF module builds work is > the way to go. Unfortunately, this means that we will not be able to use > GCC for BPF (which Daniel has been working on), because targeting the > frontend and the rest of the compiler to different architectures is only > supported by clang/LLVM. > > I just did a test build of LLVM with reduced features and targetting > only BPF, and it takes around 42 minutes to build on my 2018 macbook. > That's more than the time needed to build a typical basic OpenWrt build > from scratch. > > The way I see it, these are our options for eBPF support: > > 1. Add a host dependency on a recent enough LLVM version for eBPF. > 1a) disable it by default > 1b) hide eBPF packages unless host support is available > 2. Add llvm to tools/ to ensure that it is only built once, even when > switching between targets > 3. Add llvm as a host package and use build-dependencies in eBPF > 4. Ship precompiled big and little endian eBPF binaries + scripts to > recompile in packages, and rely on CO-RE (compile once, run everywhere) > > What do you think? > > - Felix > > _______________________________________________ > openwrt-devel mailing list > openwrt-devel@lists.openwrt.org > https://lists.openwrt.org/mailman/listinfo/openwrt-devel _______________________________________________ openwrt-devel mailing list openwrt-devel@lists.openwrt.org https://lists.openwrt.org/mailman/listinfo/openwrt-devel
