On Tue, 16 May 2023 13:18:40 -0600 Philip Prindeville <philipp_s...@redfish-solutions.com> wrote:
> Hi, > > I'm a packaging maintainer for some of the OpenWrt ancillary packages, and > I'm looking at upstreaming DPDK support into OpenWrt. Apologies in advance > if this is a bit dense (a brain dump) or hops between too many topics. > > I was looking at what's been done to date, and it seems there are a few > shortcomings which I hope to address. > > Amongst them are: > > * getting DPDK support into OpenWrt's main repo for the kmod's and into the > packages repo for the user-space support; DPDK kernel modules are deprecated, creating more usage of them is problematic. > > * making DPDK supported on all available architectures (i.e. agnostic, not > just x86_64 specific); > > * integrating into the OpenWrt "make menuconfig" system, so that editing > packages directly isn't required; Does openwrt build system support meson? > * supporting cross-building and avoiding the [flawed] assumption that the > micro-architecture of the build server is in any way related to the processor > type of the target machine(s); > > * integration into the OpenWrt CI/CD tests; > > * making the kernel support as secure/robust as possible (i.e. avoiding an > ill-behaved application taking down the kernel, since this is a firewall > after all); Not a problem with vfio > > * avoiding conflict with other existing module or package functionality; > > * avoiding, to the extent possible, introducing one-off toolchain > dependencies that unnecessarily complicate the build ecosystem; > > To this end, I'm asking the mailing list for guidance on the best packaging > practices that satisfy these goals. I'm willing to do whatever heavy lifting > that's within my ability. > > I have some related questions. > > 1. OpenWrt already bakes "vfio.ko" and "vfio-pci.ko" here: > > https://github.com/openwrt/openwrt/blob/master/package/kernel/linux/modules/virt.mk#L77-L117 > > but does so with "CONFIG_VFIO_PCI_IGD=y", which seems to be specifically > for VGA acceleration of guests in a virtualized environment. That seems to > be an odd corner case, and unlikely given that OpenWrt almost always runs on > headless hardware. Should this be reverted? Yes. > > 2. "vfio.ko" is built with "CONFIG_VFIO_NOIOMMU=y", which seems insecure. > Can this be reverted? No. Most of OpenWrt's systems do not have an IOMMU. > 3. Is "uio_pci_generic.ko" worth the potential insecurity/instability of a > misbehaving application? My understanding is that it's only needed on SR-IOV > hardware where MSI/MSI-X interrupts aren't supported: is there even any > current hardware that doesn't support MSI/MSI-X? Or am I misunderstanding > the use case? Use VFIO noiommu, it is more supported and tested upstream. With PCI generic, no interrupts work. > 4. Can most functionality be achieved with VFIO + IOMMU support? Yes. > 5. I've seen packaging for the "iommu_v2.ko" module done here: > > https://github.com/k13132/openwrt-dpdk/blob/main/packages/kmod-iommu_v2/Makefile#L22-L42 > > Is this potentially useful? What platforms/architectures use this driver? > > 6. Hand editing a wrapper for dpdk.tar.gz is a non-starter. I'd rather add > Kconfig adornments to OpenWrt packaging for the wrapper so that options for > "-Denable_drivers=" and "-Dcpu_instruction_set=" can be passed in once global > build options for OpenWrt have been selected. Defaulting the instruction set > to the build host is going to be wrong at least some of the time, if not most > of the time. For x86_64, what is a decent compromise for a > micro-architecture that will build and run on most AMD and Intel hardware? > What's a decent baseline for an ARM64 micro-architecture that will build and > run on most ARM hardware? > > 7. Many embedded systems don't build with glibc because it's too bloated (and > because critical fixes sometimes take too long to roll out), and instead use > MUSL, eglibc, or uClibc (although the last one seems to be waning). Only > glibc supports <execinfo.h> from what I can tell. Can broader support for > other C runtimes be added? Can RTE_BACKTRACE be made a parameter or > conditionally defined based on the runtime headers? (autotools would be and > HAVE_EXECINFO_H would be really handy here, but I'm not sure how to make this > play well with meson/ninja, and truth be told I'm an old school Makefile + > autotools knuckle-dragger). You could do without backtrace, but then if DPDK applications you are flying blind. > 8. How do I validate that DPDK is properly being built with the cross-tools > and not native tools? Even when building x86_64 targets on an x86_64 build > host, we end up using a custom toolchain and not the "native" compiler that > comes with the distro. > > 9. What is the parity between AMD64 and ARM64? Do both platforms offer equal > functionality and security, if not performance? Apples/Oranges. > 10. Who else is using DPDK with OpenWrt that is open to collaboration? > > 11. What is the user-space TCP/IP stack of choice (or reference) for use with > DPDK? No user-space TCP/IP stack is really robust and that great. VPP has one but it is likely to be specific to using VPP and not sure if you want to go there. I don't think Fedora/Suse/Debian/Ubuntu have packaged any userspace TCP yet.
