This roadmap has been discussed on Windows community calls Aug 5 and Aug 19
and it met no principal objections. It extends beyond the nearest release,
but it is useful to have a comprehensive backlog in one place.
In 21.11 it looks realistic to have:
* static build of DPDK installed and consumed without issues (1.1)
- the new threading API will be the top priority (1.1.4, 1.1.5)
* interrupt support for netuio (2)
* virtio PMD (3.1)
* vmxnet3 on VMWare hypervisor (3.2)
* unit tests (4.3)
* internal refactoring done (4.1, 4.2)
It is expected to at least make some progress:
* supporting cross-DLL variables, including thread-local (1.2.1)
* establishing a process to provide kernel-mode driver builds (1.3)
---
1. General Availability of DPDK on Windows
Anyone who tries to build, install, and consume DPDK on Windows hits more
or less manageable issues. The goal is to make the process friction-less.
1.1. Level 1: static userspace libraries
The last Unix-specific bits must be removed from public headers.
The hardest part is to replace pthread shim with the new threading API.
1.1.1. Remove sys/queue.h [sysq] (William Tu).
1.1.2. Remove sched.h (Dmitry Kozlyuk).
1.1.3. Fix missing exports (DmitryK).
1.1.4. New threading API [thread1, thread2] (Narcisa Ana Maria Vasile).
1.1.5. Meson < 0.59.1 cannot consume libdpdk.pc [meson1] (solved).
1.1.6. Meson >= 0.58 cannot build DPDK [meson2] (DmitryK to track).
1.1.7. Try to make Npcap ship a pkg-config file [npcap] (DmitryK, optional).
1.1.8. Rename `min`, `max`, `s_addr` [deprecation] (DmitryK, required).
1.2. Level 2: shared userspace libraries
Dynamic build is required for DPDK to become a shared Windows component.
GCC and clang take very different approach to implementing thread-local
storage; there's a risk that only clang can be supported for now.
1.2.1. Support cross-DLL variables, including thread-local (Tyler Retzlaff).
1.2.2. Reconsider embedding getopt into EAL, unsuitable for shared build.
1.3. Level 3: kernel-mode drivers
Currently netuio and virt2phys must be built manually, which
requires environment setup, and driver signature check must be disabled
to load them, which is unsuitable for production environment.
The goal is to have a contiguous process to publish latest driver builds
(unsigned) and to sign and publish selected stable versions.
There is a large organizational part in this task.
Microsoft will drive the effort, Yan Vugenfirer has volunteered to help.
virt2phys must undergo a security hardening to be signed [virt2phys].
2. Interrupt support (DmitryK - development, Intel - help with testing)
Goal: to enable usage of DPDK for burst-oriented tasks (Boulder Imaging
case) and CPU-constrained environment (no cores to dedicate for polling).
Intel repeatedly expressed interest, Microsoft will need this eventually.
3. NIC support
3.1. Enable net/virtio (and net/ena?)
Windows guests will be able to use the most widespread
paravirtual devices on Linux hosts (William Tu).
3.2. Fix netuio for net/vmxnet3 on VMWare hypervisor
The netuio kernel driver fails to load for vmxnet3 devices in VMWare
hypervisor, while it works in QEMU. Microsoft will investigate the
cause that is traced to be in Windows kernel outside of netuio.
3.3. Make Npcap provide a pkg-config file
Npcap does not provide a pkg-config file, while modern libpcap does.
DmitryK will work with Npcap developers to add it [npcap].
4. Miscellaneous
4.1. Refactor bus/pci (DmitryK)
Currently netuio maps BARs at PCI bus scan. This is against DPDK model
and can break certain scenarios, like running apps with different sets of
available PMDs. PCI bus driver code must be refactored.
4.2. Move from Setup API to cfgmgr32 API in EAL and bus/pci (DmitryK)
Setup API is deprecated, also Tyler indicated it causes layering
violation warnings by Microsoft internal tools. To make DPDK a good
Windows citizen, cfgmgr32 should be used to work with devices and driver
interfaces.
4.3. Enable unit tests [unit] (Jie Zhou)
4.5. Add WPP(?) tracing to netuio
Debugging netuio loading issues in VMWare case appeared to be impossible
until adding debug prints to pinpoint the place of failure. Tracing all
errors at least would help. virt2phys is also getting tracing in the
recent patch series. End-users can collect traces and attach them to
bug reports, this should be added to DPDK user documentation.
It is unclear who will do this and when, but Microsoft approves the
idea and can provide guidance on tracing technologies.
4.6. Find a way to install netuio without GUI.
Currently in requires Device Manager GUI, which is absent in WS Core,
it is also inconvenient to lack commands for copy&paste.
Investigate [devmgr].
References:
[sysq]: http://patchwork.dpdk.org/project/dpdk/list/?series=18438
[thread1]: http://patchwork.dpdk.org/project/dpdk/list/?series=18360
[thread2]: http://patchwork.dpdk.org/project/dpdk/list/?series=18338
[meson1]: https://github.com/mesonbuild/meson/issues/9091
[meson2]: https://github.com/mesonbuild/meson/issues/8981
[npcap]: https://github.com/nmap/npcap/issues/299
[deprecation]:
http://git.dpdk.org/dpdk/commit/?id=c4379ee599ef2f6feea914566bd494f66f7e8ad0
[unit]: http://patchwork.dpdk.org/project/dpdk/list/?series=18337
[devmgr]: https://stackoverflow.com/a/56848828/3398329
