Hi, Jan 1. Let's suppose we have the one port NIC, PCI BDF 08:00.0, SR-IOV enabled, 0 VFs is configured, netdev name is ens1f0 (depends on distro and kernel version).
2. Then, we configure 3 VFs with SR-IOV, we get : - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, ens1f1 - 08:00.2 - VF1, ens1f2 - 08:00.3 - VF2, ens1f3 3. Then, we should unbind VFs ONLY, PF should be kept untouched. - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, - - 08:00.2 - VF1, - - 08:00.3 - VF2, - 4. Then, configure switchdev mode - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, - - 08:00.2 - VF1, - - 08:00.3 - VF2, - + we get 3 representor netdevs: ens1f0_0 (no dedicated PCI device) ens1f0_1 (no dedicated PCI device) ens1f0_2 (no dedicated PCI device) 5. Then, bind VFs back (assume we are no going to map them to VMs) - 08:00.0 - PF, ens1f0 - 08:00.1 - VF0, ens1f1 - 08:00.2 - VF1, ens1f2 - 08:00.3 - VF2, ens1f3 - ens1f0_0 (no dedicated PCI device) - ens1f0_1 (no dedicated PCI device) - ens1f0_2 (no dedicated PCI device) 6. As result we should get 7 netdevs - 1 PF (Uplink representor), 3 VFs, 3 representors (no PCI device, pure netdev) With best regards, Slava > -----Original Message----- > From: Jan Viktorin <vikto...@cesnet.cz> > Sent: Monday, March 15, 2021 18:11 > To: Slava Ovsiienko <viachesl...@nvidia.com> > Cc: dev@dpdk.org; Asaf Penso <as...@nvidia.com>; Shahaf Shuler > <shah...@nvidia.com>; Matan Azrad <ma...@nvidia.com> > Subject: Re: [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev > mode > > Hello Salva, > > On Mon, 15 Mar 2021 15:53:51 +0000 > Slava Ovsiienko <viachesl...@nvidia.com> wrote: > > > Hi, Jan > > > > Yes, bullet [4] explicitly requires to unbind VFs, and detach the netdevs > from the mlx5_core driver. > > Otherwise, kernel driver refuses to be configured with switchdev mode in > [5]. So, [4] can't be skipped. > > After setting swithdev mode, VFs can be bound back (if it is needed, and > these ones are not mapped to VMs): > > OK, but I believe that it is **not possible** to follow the rule [5]. > The guide explicitly says in [4] "can be rebind **after** the switchdev > mode". > Just, if you unbind the device, there is no way how to configure the > switchdev mode, this is the contradiction I mentioned in the commit. You > cannot configure switchdev mode because the interface is gone and the path > /sys/class/net/<net device>/compat/devlink/mode no longer exists. > > So, maybe, just the formulation is wrong. So, what is the **exact > right** way how to do it? I would change the commit accordingly. Just, let's > make it right. Would it work this way? > > # echo -n "<device pci address>" > /sys/bus/pci/drivers/mlx5_core/unbind > # echo -n "<device pci address>" > /sys/bus/pci/drivers/mlx5_core/bind > # echo switchdev > /sys/class/net/<net device>/compat/devlink/mode > > It is good to mention that after the rebind, the <net device> can change. > > Regards, > Jan > > > > > echo -n "<device pci address>" > > /sys/bus/pci/drivers/mlx5_core/bind > > > > With best regards, > > Slava > > > > > -----Original Message----- > > > From: Jan Viktorin <ivikto...@fit.vutbr.cz> > > > Sent: Monday, March 15, 2021 17:34 > > > To: dev@dpdk.org > > > Cc: Jan Viktorin <vikto...@cesnet.cz>; Asaf Penso > > > <as...@nvidia.com>; Shahaf Shuler <shah...@nvidia.com>; Slava > > > Ovsiienko <viachesl...@nvidia.com>; Matan Azrad > <ma...@nvidia.com> > > > Subject: [PATCH 2/2] net/mlx5: avoid unbind step to enable switchdev > > > mode > > > > > > From: Jan Viktorin <vikto...@cesnet.cz> > > > > > > The step 4 is a contradiction. It advices to unbind the device from > > > the mlx5_core which removes the associated system network interface > (e.g. > > > eth0). In the step 5, the same system network interface (e.g. eth0) > > > is required to exist. > > > > > > Signed-off-by: Jan Viktorin <vikto...@cesnet.cz> > > > --- > > > doc/guides/nics/mlx5.rst | 6 +----- > > > 1 file changed, 1 insertion(+), 5 deletions(-) > > > > > > diff --git a/doc/guides/nics/mlx5.rst b/doc/guides/nics/mlx5.rst > > > index 0a2dc3dee..122d8e0fc 100644 > > > --- a/doc/guides/nics/mlx5.rst > > > +++ b/doc/guides/nics/mlx5.rst > > > @@ -1370,11 +1370,7 @@ the DPDK application. > > > > > > echo <num of vfs > /sys/class/net/<net > > > device>/device/sriov_numvfs > > > > > > -4. Unbind the device (can be rebind after the switchdev mode):: > > > - > > > - echo -n "<device pci address>" > > > > /sys/bus/pci/drivers/mlx5_core/unbind > > > - > > > -5. Enable switchdev mode:: > > > +4. Enable switchdev mode:: > > > > > > echo switchdev > /sys/class/net/<net > > > device>/compat/devlink/mode > > > > > > -- > > > 2.30.1 > >
