On 4/13/2023 8:59 AM, Maxime Coquelin wrote: > Hi, > > On 4/13/23 09:08, Xia, Chenbo wrote: >>> -----Original Message----- >>> From: Morten Brørup <m...@smartsharesystems.com> >>> Sent: Thursday, April 13, 2023 3:41 AM >>> To: Maxime Coquelin <maxime.coque...@redhat.com>; Ferruh Yigit >>> <ferruh.yi...@amd.com>; dev@dpdk.org; david.march...@redhat.com; Xia, >>> Chenbo <chenbo....@intel.com>; m...@redhat.com; f...@redhat.com; >>> jasow...@redhat.com; Liang, Cunming <cunming.li...@intel.com>; Xie, >>> Yongji >>> <xieyon...@bytedance.com>; echau...@redhat.com; epere...@redhat.com; >>> amore...@redhat.com >>> Subject: RE: [RFC 00/27] Add VDUSE support to Vhost library >>> >>>> From: Maxime Coquelin [mailto:maxime.coque...@redhat.com] >>>> Sent: Wednesday, 12 April 2023 17.28 >>>> >>>> Hi Ferruh, >>>> >>>> On 4/12/23 13:33, Ferruh Yigit wrote: >>>>> On 3/31/2023 4:42 PM, Maxime Coquelin wrote: >>>>>> This series introduces a new type of backend, VDUSE, >>>>>> to the Vhost library. >>>>>> >>>>>> VDUSE stands for vDPA device in Userspace, it enables >>>>>> implementing a Virtio device in userspace and have it >>>>>> attached to the Kernel vDPA bus. >>>>>> >>>>>> Once attached to the vDPA bus, it can be used either by >>>>>> Kernel Virtio drivers, like virtio-net in our case, via >>>>>> the virtio-vdpa driver. Doing that, the device is visible >>>>>> to the Kernel networking stack and is exposed to userspace >>>>>> as a regular netdev. >>>>>> >>>>>> It can also be exposed to userspace thanks to the >>>>>> vhost-vdpa driver, via a vhost-vdpa chardev that can be >>>>>> passed to QEMU or Virtio-user PMD. >>>>>> >>>>>> While VDUSE support is already available in upstream >>>>>> Kernel, a couple of patches are required to support >>>>>> network device type: >>>>>> >>>>>> https://gitlab.com/mcoquelin/linux/-/tree/vduse_networking_poc >>>>>> >>>>>> In order to attach the created VDUSE device to the vDPA >>>>>> bus, a recent iproute2 version containing the vdpa tool is >>>>>> required. >>>>> >>>>> Hi Maxime, >>>>> >>>>> Is this a replacement to the existing DPDK vDPA framework? What is the >>>>> plan for long term? >>>>> >>>> >>>> No, this is not a replacement for DPDK vDPA framework. >>>> >>>> We (Red Hat) don't have plans to support DPDK vDPA framework in our >>>> products, but there are still contribution to DPDK vDPA by several vDPA >>>> hardware vendors (Intel, Nvidia, Xilinx), so I don't think it is going >>>> to be deprecated soon. >>> >>> Ferruh's question made me curious... >>> >>> I don't know anything about VDUSE or vDPA, and don't use any of it, so >>> consider me ignorant in this area. >>> >>> Is VDUSE an alternative to the existing DPDK vDPA framework? What are >>> the >>> differences, e.g. in which cases would an application developer (or >>> user) >>> choose one or the other? >> >> Maxime should give better explanation.. but let me just explain a bit. >> >> Vendors have vDPA HW that support vDPA framework (most likely in their >> DPU/IPU >> products). This work is introducing a way to emulate a SW vDPA device in >> userspace (DPDK), and this SW vDPA device also supports vDPA framework. >> >> So it's not an alternative to existing DPDK vDPA framework :) > > Correct. > > When using DPDK vDPA, the datapath of a Vhost-user port is offloaded to > a compatible physical NIC (i.e. a NIC that implements Virtio rings > support), the control path remains the same as a regular Vhost-user > port, i.e. it provides a Vhost-user unix socket to the application (like > QEMU or DPDK Virtio-user PMD). > > When using Kernel vDPA, the datapath is also offloaded to a vDPA > compatible device, and the control path is managed by the vDPA bus. > It can either be consumed by a Kernel Virtio device (here Virtio-net) > when using Virtio-vDPA. In this case the device is exposed as a regular > netdev and, in the case of Kubernetes, can be used as primary interfaces > for the pods. > Or it can be exposed to user-space via Vhost-vDPA, a chardev that can be > seen as an alternative to Vhost-user sockets. In this case it can for > example be used by QEMU or DPDK Virtio-user PMD. In Kubernetes, it can > be used as a secondary interface. > > Now comes VDUSE. VDUSE is a Kernel vDPA device, but instead of being a > physical device where the Virtio datapath is offloaded, the Virtio > datapath is offloaded to a user-space application. With this series, a > DPDK application, like OVS-DPDK for instance, can create VDUSE device > and expose them either as regular netdev when binding them to Kernel > Virtio-net driver via Virtio-vDPA, or as Vhost-vDPA interface to be > consumed by another userspace appliation like QEMU or DPDK application > using Virtio-user PMD. With this solution, OVS-DPDK could serve both > primary and secondary interfaces of Kubernetes pods. > > I hope it clarifies, I will add these information in the cover-letter > for next revisions. Let me know if anything is still unclear. > > I did a presentation at last DPDK summit [0], maybe the diagrams will > help to clarify furthermore. >
Thanks Chenbo, Maxime for clarification. After reading a little more (I think) I got it better, slides [0] were useful. So this is more like a backend/handler, similar to vhost-user, although it is vDPA device emulation. Can you please describe more the benefit of vduse comparing to vhost-user? Also what is "VDUSE daemon", which is referred a few times in documentation, is it another userspace implementation of the vduse? >>> >>> And if it is a better alternative, perhaps the documentation should >>> mention that it is recommended over DPDK vDPA. Just like we started >>> recommending alternatives to the KNI driver, so we could phase it out >>> and >>> eventually get rid of it. >>> >>>> >>>> Regards, >>>> Maxime >> > > [0]: > https://static.sched.com/hosted_files/dpdkuserspace22/9f/Open%20DPDK%20to%20containers%20networking%20with%20VDUSE.pdf >
