Hi Parav, > -----Original Message----- > From: Parav Pandit <pa...@nvidia.com> > Sent: Tuesday, June 15, 2021 1:43 PM > To: Xia, Chenbo <chenbo....@intel.com>; NBU-Contact-Thomas Monjalon > <tho...@monjalon.net>; Yigit, Ferruh <ferruh.yi...@intel.com> > Cc: Xueming(Steven) Li <xuemi...@nvidia.com>; Slava Ovsiienko > <viachesl...@nvidia.com>; dev@dpdk.org; Jason Gunthorpe <j...@nvidia.com> > Subject: RE: [dpdk-dev] [RFC 00/14] mlx5: support SubFunction > > > > > From: Xia, Chenbo <chenbo....@intel.com> > > Sent: Tuesday, June 15, 2021 11:03 AM > > > > Hi Parav, > > > > > -----Original Message----- > > > From: Parav Pandit <pa...@nvidia.com> > > > Sent: Tuesday, June 15, 2021 12:05 PM > > > To: Xia, Chenbo <chenbo....@intel.com>; NBU-Contact-Thomas Monjalon > > > <tho...@monjalon.net>; Yigit, Ferruh <ferruh.yi...@intel.com> > > > Cc: Xueming(Steven) Li <xuemi...@nvidia.com>; Slava Ovsiienko > > > <viachesl...@nvidia.com>; dev@dpdk.org; Jason Gunthorpe > > > <j...@nvidia.com> > > > Subject: RE: [dpdk-dev] [RFC 00/14] mlx5: support SubFunction > > > > > > Hi Chenbo, > > > > > > > From: Xia, Chenbo <chenbo....@intel.com> > > > > Sent: Tuesday, June 15, 2021 7:41 AM > > > > > > > > Hi Thomas, > > > > > > > > > From: Thomas Monjalon <tho...@monjalon.net> > > > > > Sent: Friday, June 11, 2021 3:54 PM > > > [..] > > > > > > > > > > > Yes. In our term it's called Assignable Device Interface (ADI) > > > > introduced in Intel Scalable IOV > > > > (https://01.org/blogs/2019/assignable-interfaces-intel- > > > > scalable-i/o-virtualization-linux) > > > > > > > > And vfio-mdev is chosen to be the software framework for it. I start > > > > to > > > realize > > > > there is difference between SF and ADI: SF considers multi-function > > > > devices which may include net/regex/vdpa/... > > > Yes. net, rdma, vdpa, regex ++. > > > And eventually vfio_device to map to VM too. > > > > > > Non mdev framework is chosen so that all the use cases of kernel only, > > > or user only or mix modes can be supported. > > > > OK. Got it. > > > > > > > > > But ADI only focuses on the > > > > virtualization of the devices and splitting devices to logic parts > > > > and > > > providing > > > > huge number of interfaces to host APP. I think SF also considers > > > > this but is mainly used for multi-function devices (like DPU in your > term? > > > > Correct me if I'm wrong). > > > > > > > SF also supports DPU mode too but it is in addition to above use cases. > > > SF will expose mdev (or a vfio_device) to map to a VM. > > > > So your SW actually supports vfio-mdev? I suppose the device-specific mdev > > Kernel module is out-of-tree? > > > mlx5 driver doesn't support vfio_device for SFs. > Kernel plumbing for PASID assignment to SF is WIP currently kernel community. > We do not have any out-of-tree kernel module. > > > Just FYI: > > > > We are introducing a new mdev bus for DPDK: > > http://patchwork.dpdk.org/project/dpdk/cover/20210601030644.3318-1- > > chenbo....@intel.com/ > > > I am yet to read about it. But I am not sure what value does it add. > A user can open a vfio device using vfio subsystem and operate on it. > A vfio device can be a create as a result of binding PCI VF/PF to vfio-pci > driver or a SF by binding SF to vfio_foo driver.
Yes, in general it is the way. For vfio-mdev, it works as binding the vfio-mdev to parent device and echo uuid to create a virtual device. VFIO APP like DPDK, as you said, should work similar with VFIO UAPI for vfio-pci devices or mdev-based devices. But currently DPDK only cares about vfio-pci devices and does not care things for other cases like mdev-based pci devices. For example, it does not scan /sys/bus/mdev and it always uses pci bdf as device address, which mdev-based pci devices do not have. Therefore I sent that patchset. > There is kernel work in progress to use vfio core as library. OK. Could you share me some link to it? Much appreciated. > So we do not anticipate to use add mdev layer and uuid to create a vfio device > for a SF. OK. For now, we are following the vfio-mdev standard, using UUID to create vfio devices. > > For Intel, ADI will never has any netdevs or rdma dev? I think technically it could have. But for some devices like our dma devices, it's just using mdev: https://www.spinics.net/lists/kvm/msg244417.html Thanks, Chenbo
