On Thu, Jun 8, 2023 at 11:17 AM Maxime Coquelin <maxime.coque...@redhat.com> wrote: > On 6/7/23 10:05, David Marchand wrote: > > On Tue, Jun 6, 2023 at 10:19 AM Maxime Coquelin > > <maxime.coque...@redhat.com> wrote: > >> > >> This series introduces a new type of backend, VDUSE, > >> to the Vhost library. > >> > >> VDUSE stands for vDPA device in Userspace, it enables > >> implementing a Virtio device in userspace and have it > >> attached to the Kernel vDPA bus. > >> > >> Once attached to the vDPA bus, it can be used either by > >> Kernel Virtio drivers, like virtio-net in our case, via > >> the virtio-vdpa driver. Doing that, the device is visible > >> to the Kernel networking stack and is exposed to userspace > >> as a regular netdev. > >> > >> It can also be exposed to userspace thanks to the > >> vhost-vdpa driver, via a vhost-vdpa chardev that can be > >> passed to QEMU or Virtio-user PMD. > >> > >> While VDUSE support is already available in upstream > >> Kernel, a couple of patches are required to support > >> network device type: > >> > >> https://gitlab.com/mcoquelin/linux/-/tree/vduse_networking_rfc > >> > >> In order to attach the created VDUSE device to the vDPA > >> bus, a recent iproute2 version containing the vdpa tool is > >> required. > >> > >> Benchmark results: > >> ================== > >> > >> On this v2, PVP reference benchmark has been run & compared with > >> Vhost-user. > >> > >> When doing macswap forwarding in the worload, no difference is seen. > >> When doing io forwarding in the workload, we see 4% performance > >> degradation with VDUSE, comapred to Vhost-user/Virtio-user. It is > >> explained by the use of the IOTLB layer in the Vhost-library when using > >> VDUSE, whereas Vhost-user/Virtio-user does not make use of it. > >> > >> Usage: > >> ====== > >> > >> 1. Probe required Kernel modules > >> # modprobe vdpa > >> # modprobe vduse > >> # modprobe virtio-vdpa > >> > >> 2. Build (require vduse kernel headers to be available) > >> # meson build > >> # ninja -C build > >> > >> 3. Create a VDUSE device (vduse0) using Vhost PMD with > >> testpmd (with 4 queue pairs in this example) > >> # ./build/app/dpdk-testpmd --no-pci > >> --vdev=net_vhost0,iface=/dev/vduse/vduse0,queues=4 --log-level=*:9 -- -i > >> --txq=4 --rxq=4 > > > > 9 is a nice but undefined value. 8 is enough. > > In general, I prefer "human readable" strings, like *:debug ;-). > > > > > >> > >> 4. Attach the VDUSE device to the vDPA bus > >> # vdpa dev add name vduse0 mgmtdev vduse > >> => The virtio-net netdev shows up (eth0 here) > >> # ip l show eth0 > >> 21: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP > >> mode DEFAULT group default qlen 1000 > >> link/ether c2:73:ea:a7:68:6d brd ff:ff:ff:ff:ff:ff > >> > >> 5. Start/stop traffic in testpmd > >> testpmd> start > >> testpmd> show port stats 0 > >> ######################## NIC statistics for port 0 > >> ######################## > >> RX-packets: 11 RX-missed: 0 RX-bytes: 1482 > >> RX-errors: 0 > >> RX-nombuf: 0 > >> TX-packets: 1 TX-errors: 0 TX-bytes: 62 > >> > >> Throughput (since last show) > >> Rx-pps: 0 Rx-bps: 0 > >> Tx-pps: 0 Tx-bps: 0 > >> > >> ############################################################################ > >> testpmd> stop > >> > >> 6. Detach the VDUSE device from the vDPA bus > >> # vdpa dev del vduse0 > >> > >> 7. Quit testpmd > >> testpmd> quit > >> > >> Known issues & remaining work: > >> ============================== > >> - Fix issue in FD manager (still polling while FD has been removed) > >> - Add Netlink support in Vhost library > >> - Support device reconnection > >> -> a temporary patch to support reconnection via a tmpfs file is > >> available, > >> upstream solution would be in-kernel and is being developed. > >> -> > >> https://gitlab.com/mcoquelin/dpdk-next-virtio/-/commit/5ad06ce14159a9ce36ee168dd13ef389cec91137 > >> - Support packed ring > >> - Provide more performance benchmark results > > > > We are missing a reference to the kernel patches required to have > > vduse accept net devices. > > Right, I mention it in the cover letter, but it should be in the release > note also. I propose to append this to the release note: > "While VDUSE support is already available in upstream Kernel, a couple > of patches are required to support network device type, which are being > upstreamed: > https://lore.kernel.org/all/20230419134329.346825-1-maxime.coque...@redhat.com/"; > > Does that sound good to you?
Ok for me. Thanks. -- David Marchand
