On 4/14/23 12:48, Ferruh Yigit wrote:
On 4/13/2023 8:59 AM, Maxime Coquelin wrote:
Hi,
On 4/13/23 09:08, Xia, Chenbo wrote:
-----Original Message-----
From: Morten Brørup <m...@smartsharesystems.com>
Sent: Thursday, April 13, 2023 3:41 AM
To: Maxime Coquelin <maxime.coque...@redhat.com>; Ferruh Yigit
<ferruh.yi...@amd.com>; dev@dpdk.org; david.march...@redhat.com; Xia,
Chenbo <chenbo....@intel.com>; m...@redhat.com; f...@redhat.com;
jasow...@redhat.com; Liang, Cunming <cunming.li...@intel.com>; Xie,
Yongji
<xieyon...@bytedance.com>; echau...@redhat.com; epere...@redhat.com;
amore...@redhat.com
Subject: RE: [RFC 00/27] Add VDUSE support to Vhost library
From: Maxime Coquelin [mailto:maxime.coque...@redhat.com]
Sent: Wednesday, 12 April 2023 17.28
Hi Ferruh,
On 4/12/23 13:33, Ferruh Yigit wrote:
On 3/31/2023 4:42 PM, Maxime Coquelin wrote:
This series introduces a new type of backend, VDUSE,
to the Vhost library.
VDUSE stands for vDPA device in Userspace, it enables
implementing a Virtio device in userspace and have it
attached to the Kernel vDPA bus.
Once attached to the vDPA bus, it can be used either by
Kernel Virtio drivers, like virtio-net in our case, via
the virtio-vdpa driver. Doing that, the device is visible
to the Kernel networking stack and is exposed to userspace
as a regular netdev.
It can also be exposed to userspace thanks to the
vhost-vdpa driver, via a vhost-vdpa chardev that can be
passed to QEMU or Virtio-user PMD.
While VDUSE support is already available in upstream
Kernel, a couple of patches are required to support
network device type:
https://gitlab.com/mcoquelin/linux/-/tree/vduse_networking_poc
In order to attach the created VDUSE device to the vDPA
bus, a recent iproute2 version containing the vdpa tool is
required.
Hi Maxime,
Is this a replacement to the existing DPDK vDPA framework? What is the
plan for long term?
No, this is not a replacement for DPDK vDPA framework.
We (Red Hat) don't have plans to support DPDK vDPA framework in our
products, but there are still contribution to DPDK vDPA by several vDPA
hardware vendors (Intel, Nvidia, Xilinx), so I don't think it is going
to be deprecated soon.
Ferruh's question made me curious...
I don't know anything about VDUSE or vDPA, and don't use any of it, so
consider me ignorant in this area.
Is VDUSE an alternative to the existing DPDK vDPA framework? What are
the
differences, e.g. in which cases would an application developer (or
user)
choose one or the other?
Maxime should give better explanation.. but let me just explain a bit.
Vendors have vDPA HW that support vDPA framework (most likely in their
DPU/IPU
products). This work is introducing a way to emulate a SW vDPA device in
userspace (DPDK), and this SW vDPA device also supports vDPA framework.
So it's not an alternative to existing DPDK vDPA framework :)
Correct.
When using DPDK vDPA, the datapath of a Vhost-user port is offloaded to
a compatible physical NIC (i.e. a NIC that implements Virtio rings
support), the control path remains the same as a regular Vhost-user
port, i.e. it provides a Vhost-user unix socket to the application (like
QEMU or DPDK Virtio-user PMD).
When using Kernel vDPA, the datapath is also offloaded to a vDPA
compatible device, and the control path is managed by the vDPA bus.
It can either be consumed by a Kernel Virtio device (here Virtio-net)
when using Virtio-vDPA. In this case the device is exposed as a regular
netdev and, in the case of Kubernetes, can be used as primary interfaces
for the pods.
Or it can be exposed to user-space via Vhost-vDPA, a chardev that can be
seen as an alternative to Vhost-user sockets. In this case it can for
example be used by QEMU or DPDK Virtio-user PMD. In Kubernetes, it can
be used as a secondary interface.
Now comes VDUSE. VDUSE is a Kernel vDPA device, but instead of being a
physical device where the Virtio datapath is offloaded, the Virtio
datapath is offloaded to a user-space application. With this series, a
DPDK application, like OVS-DPDK for instance, can create VDUSE device
and expose them either as regular netdev when binding them to Kernel
Virtio-net driver via Virtio-vDPA, or as Vhost-vDPA interface to be
consumed by another userspace appliation like QEMU or DPDK application
using Virtio-user PMD. With this solution, OVS-DPDK could serve both
primary and secondary interfaces of Kubernetes pods.
I hope it clarifies, I will add these information in the cover-letter
for next revisions. Let me know if anything is still unclear.
I did a presentation at last DPDK summit [0], maybe the diagrams will
help to clarify furthermore.
Thanks Chenbo, Maxime for clarification.
After reading a little more (I think) I got it better, slides [0] were
useful.
So this is more like a backend/handler, similar to vhost-user, although
it is vDPA device emulation.
Can you please describe more the benefit of vduse comparing to vhost-user?
The main benefit is that VDUSE device can be exposed as a regular
netdev, while this is not possible with Vhost-user.
Also what is "VDUSE daemon", which is referred a few times in
documentation, is it another userspace implementation of the vduse?
VDUSE daemon is the application that implements the VDUSE device, e.g.
OVS-DPDK with DPDK Vhost library using this series in our case.
Maxime
And if it is a better alternative, perhaps the documentation should
mention that it is recommended over DPDK vDPA. Just like we started
recommending alternatives to the KNI driver, so we could phase it out
and
eventually get rid of it.
Regards,
Maxime
[0]:
https://static.sched.com/hosted_files/dpdkuserspace22/9f/Open%20DPDK%20to%20containers%20networking%20with%20VDUSE.pdf
