Hello Ferruh, Thanks for your feedback.
On 2023-02-15 13:37:05 +0000, Ferruh Yigit wrote: > On 2/3/2023 8:08 AM, Chaoyong He wrote: > > From: Walter Heymans <walter.heym...@corigine.com> > > > > The NFP PMD documentation is updated to include information about > > Corigine and their new vendor device ID. > > > > Outdated information regarding the use of the PMD is also updated. > > > > While making major changes to the document, the maximum number of > > characters per line is updated to 80 characters to improve the > > readability in raw format. > > > > There are three groups of changes done to documentation as explained in > three paragraphs above. > > To help review, is it possible to separate this patch into three > patches? Later they can be squashed and merged as a single patch. > But as it is, easy to miss content changes among formatting changes. > > (You can include simple grammar updates (that doesn't change either > content or Corigine related information) to formatting update patch) We will break this patch in to three as you suggest, address the comments below and post a v2. > > > > Signed-off-by: Walter Heymans <walter.heym...@corigine.com> > > Reviewed-by: Niklas Söderlund <niklas.soderl...@corigine.com> > > Reviewed-by: Chaoyong He <chaoyong...@corigine.com> > > --- > > doc/guides/nics/nfp.rst | 168 +++++++++++++++++++++------------------- > > 1 file changed, 90 insertions(+), 78 deletions(-) > > > > diff --git a/doc/guides/nics/nfp.rst b/doc/guides/nics/nfp.rst > > index a085d7d9ae..6fea280411 100644 > > --- a/doc/guides/nics/nfp.rst > > +++ b/doc/guides/nics/nfp.rst > > @@ -1,35 +1,34 @@ > > .. SPDX-License-Identifier: BSD-3-Clause > > Copyright(c) 2015-2017 Netronome Systems, Inc. All rights reserved. > > - All rights reserved. > > + Copyright(c) 2021 Corigine, Inc. All rights reserved. > > > > NFP poll mode driver library > > ============================ > > > > -Netronome's sixth generation of flow processors pack 216 programmable > > -cores and over 100 hardware accelerators that uniquely combine packet, > > -flow, security and content processing in a single device that scales > > +Netronome and Corigine's sixth generation of flow processors pack 216 > > +programmable cores and over 100 hardware accelerators that uniquely combine > > +packet, flow, security and content processing in a single device that > > scales > > up to 400-Gb/s. > > > > -This document explains how to use DPDK with the Netronome Poll Mode > > -Driver (PMD) supporting Netronome's Network Flow Processor 6xxx > > -(NFP-6xxx), Netronome's Network Flow Processor 4xxx (NFP-4xxx) and > > -Netronome's Network Flow Processor 38xx (NFP-38xx). > > +This document explains how to use DPDK with the Network Flow Processor > > (NFP) > > +Poll Mode Driver (PMD) supporting Netronome and Corigine's NFP-6xxx, > > NFP-4xxx > > +and NFP-38xx product lines. > > > > -NFP is a SRIOV capable device and the PMD supports the physical > > -function (PF) and the virtual functions (VFs). > > +NFP is a SR-IOV capable device and the PMD supports the physical function > > (PF) > > +and the virtual functions (VFs). > > > > Dependencies > > ------------ > > > > -Before using the Netronome's DPDK PMD some NFP configuration, > > -which is not related to DPDK, is required. The system requires > > -installation of **Netronome's BSP (Board Support Package)** along > > -with a specific NFP firmware application. Netronome's NSP ABI > > -version should be 0.20 or higher. > > +Before using the NFP DPDK PMD some NFP configuration, which is not related > > to > > +DPDK, is required. The system requires installation of > > +**NFP-BSP (Board Support Package)** along with a specific NFP firmware > > +application. The NSP ABI version should be 0.20 or higher. > > > > -If you have a NFP device you should already have the code and > > -documentation for this configuration. Contact > > -**supp...@netronome.com** to obtain the latest available firmware. > > +If you have a NFP device you should already have the documentation to > > perform > > +this configuration. Contact **supp...@netronome.com** (for Netronome > > products) > > +or **smartnic-supp...@corigine.com** (for Corigine products) to obtain the > > +latest available firmware. > > > > The NFP Linux netdev kernel driver for VFs has been a part of the > > vanilla kernel since kernel version 4.5, and support for the PF > > @@ -44,11 +43,11 @@ Linux kernel driver. > > Building the software > > --------------------- > > > > -Netronome's PMD code is provided in the **drivers/net/nfp** directory. > > -Although NFP PMD has Netronome´s BSP dependencies, it is possible to > > -compile it along with other DPDK PMDs even if no BSP was installed > > previously. > > -Of course, a DPDK app will require such a BSP installed for using the > > -NFP PMD, along with a specific NFP firmware application. > > +The NFP PMD code is provided in the **drivers/net/nfp** directory. Although > > +NFP PMD has BSP dependencies, it is possible to compile it along with other > > +DPDK PMDs even if no BSP was installed previously. Of course, a DPDK app > > will > > +require such a BSP installed for using the NFP PMD, along with a specific > > NFP > > +firmware application. > > > > Once the DPDK is built all the DPDK apps and examples include support for > > the NFP PMD. > > @@ -57,27 +56,20 @@ the NFP PMD. > > Driver compilation and testing > > ------------------------------ > > > > -Refer to the document :ref:`compiling and testing a PMD for a NIC > > <pmd_build_and_test>` > > -for details. > > +Refer to the document > > +:ref:`compiling and testing a PMD for a NIC <pmd_build_and_test>` for > > details. > > > > Using the PF > > ------------ > > > > -NFP PMD supports using the NFP PF as another DPDK port, but it does not > > -have any functionality for controlling VFs. In fact, it is not possible to > > use > > -the PMD with the VFs if the PF is being used by DPDK, that is, with the > > NFP PF > > -bound to ``igb_uio`` or ``vfio-pci`` kernel drivers. Future DPDK versions > > will > > -have a PMD able to work with the PF and VFs at the same time and with the > > PF > > -implementing VF management along with other PF-only > > functionalities/offloads. > > - > > Why this paragraph is removed? Is it because it is not correct anymore, > or just because of document organization change. > > > The PMD PF has extra work to do which will delay the DPDK app > > initialization > > -like uploading the firmware and configure the Link state properly when > > starting or > > -stopping a PF port. Since DPDK 18.05 the firmware upload happens when > > +like uploading the firmware and configure the Link state properly when > > starting > > +or stopping a PF port. Since DPDK 18.05 the firmware upload happens when > > a PF is initialized, which was not always true with older DPDK versions. > > > > -Depending on the Netronome product installed in the system, firmware files > > -should be available under ``/lib/firmware/netronome``. DPDK PMD supporting > > the > > -PF looks for a firmware file in this order: > > +Depending on the product installed in the system, firmware files should be > > +available under ``/lib/firmware/netronome``. DPDK PMD supporting the PF > > looks > > +for a firmware file in this order: > > > > 1) First try to find a firmware image specific for this device using the > > NFP serial number: > > @@ -92,18 +84,21 @@ PF looks for a firmware file in this order: > > > > nic_AMDA0099-0001_2x25.nffw > > > > -Netronome's software packages install firmware files under > > ``/lib/firmware/netronome`` > > -to support all the Netronome's SmartNICs and different firmware > > applications. > > -This is usually done using file names based on SmartNIC type and media and > > with a > > -directory per firmware application. Options 1 and 2 for firmware filenames > > allow > > -more than one SmartNIC, same type of SmartNIC or different ones, and to > > upload a > > -different firmware to each SmartNIC. > > +Netronome and Corigine's software packages install firmware files under > > +``/lib/firmware/netronome`` to support all the SmartNICs and different > > firmware > > +applications. This is usually done using file names based on SmartNIC type > > and > > +media and with a directory per firmware application. Options 1 and 2 for > > +firmware filenames allow more than one SmartNIC, same type of SmartNIC or > > +different ones, and to upload a different firmware to each SmartNIC. > > > > .. Note:: > > - Currently the NFP PMD supports using the PF with Agilio Firmware > > with NFD3 > > - and Agilio Firmware with NFDk. See > > https://help.netronome.com/support/solutions > > + Currently the NFP PMD supports using the PF with Agilio Firmware with > > + NFD3 and Agilio Firmware with NFDk. See > > + `Netronome Support <https://help.netronome.com/support/solutions>`_. > > for more information on the various firmwares supported by the > > Netronome > > - Agilio CX smartNIC. > > + Agilio SmartNICs range, or > > + `Corigine Support > > <https://www.corigine.com/productsOverviewList-30.html>`_. > > + for more information about Corigine's range. > > > > PF multiport support > > -------------------- > > @@ -118,7 +113,7 @@ this particular configuration requires the PMD to > > create ports in a special way, > > although once they are created, DPDK apps should be able to use them as > > normal > > PCI ports. > > > > -NFP ports belonging to same PF can be seen inside PMD initialization with a > > +NFP ports belonging to the same PF can be seen inside PMD initialization > > with a > > suffix added to the PCI ID: wwww:xx:yy.z_portn. For example, a PF with PCI > > ID > > 0000:03:00.0 and four ports is seen by the PMD code as: > > > > @@ -137,50 +132,67 @@ suffix added to the PCI ID: wwww:xx:yy.z_portn. For > > example, a PF with PCI ID > > PF multiprocess support > > ----------------------- > > > > -Due to how the driver needs to access the NFP through a CPP interface, > > which implies > > -to use specific registers inside the chip, the number of secondary > > processes with PF > > -ports is limited to only one. > > +Due to how the driver needs to access the NFP through a CPP interface, > > which > > +implies to use specific registers inside the chip, the number of secondary > > +processes with PF ports is limited to only one. > > > > -This limitation will be solved in future versions but having basic > > multiprocess support > > -is important for allowing development and debugging through the PF using a > > secondary > > -process which will create a CPP bridge for user space tools accessing the > > NFP. > > +This limitation will be solved in future versions, but having basic > > +multiprocess support is important for allowing development and debugging > > +through the PF using a secondary process, which will create a CPP bridge > > +for user space tools accessing the NFP. > > > > > > System configuration > > -------------------- > > > > #. **Enable SR-IOV on the NFP device:** The current NFP PMD supports the > > PF and > > - the VFs on a NFP device. However, it is not possible to work with both > > at the > > - same time because the VFs require the PF being bound to the NFP PF Linux > > - netdev driver. Make sure you are working with a kernel with NFP PF > > support or > > - get the drivers from the above Github repository and follow the > > instructions > > - for building and installing it. > > + the VFs on a NFP device. However, it is not possible to work with both > > at > > + the same time when using the netdev NFP Linux netdev driver. > > Old and new text doesn't say same thing. > Old one says: "For DPDK to support VF, PF needs to bound to kernel driver.: > > Is this changed, or just wording mistake? > > > > It is possible > > + to bind the PF to the ``vfio-pci`` kernel module, and create VFs > > afterwards. > > + This requires loading the ``vfio-pci`` module with the following > > parameters: > > + > > + .. code-block:: console > > + > > + modprobe vfio-pci enable_sriov=1 disable_idle_d3=1 > > + > > + VFs need to be enabled before they can be used with the PMD. Before > > enabling > > + the VFs it is useful to obtain information about the current NFP PCI > > device > > + detected by the system. This can be done on Netronome SmartNICs using: > > + > > + .. code-block:: console > > + > > + lspci -d 19ee: > > > > What I understand is, to support VF by DPDK two things are required: > 1) Ability to create VFs, this can be done both by using device's kernel > driver or 'vfio-pci' > 2) PF driver should support managing VFs. > > Above lines document about item (1) and how 'vfio-pci' is used for it. > > But old documentation mentions about item (2) is missing, why that part > removed, isn't it valid anymore? I mean is "PF -> kernel, VF -> DPDK" > combination supported now? > > > > - VFs need to be enabled before they can be used with the PMD. > > - Before enabling the VFs it is useful to obtain information about the > > - current NFP PCI device detected by the system: > > + and on Corigine SmartNICs using: > > > > .. code-block:: console > > > > - lspci -d19ee: > > + lspci -d 1da8: > > > > - Now, for example, configure two virtual functions on a NFP-6xxx device > > + Now, for example, to configure two virtual functions on a NFP device > > whose PCI system identity is "0000:03:00.0": > > > > .. code-block:: console > > > > echo 2 > /sys/bus/pci/devices/0000:03:00.0/sriov_numvfs > > > > - The result of this command may be shown using lspci again: > > + The result of this command may be shown using lspci again on Netronome > > + SmartNICs: > > + > > + .. code-block:: console > > + > > + lspci -d 19ee: -k > > + > > + and on Corigine SmartNICs: > > > > .. code-block:: console > > > > - lspci -d19ee: -k > > + lspci -d 1da8: -k > > > > Two new PCI devices should appear in the output of the above command. > > The > > - -k option shows the device driver, if any, that devices are bound to. > > - Depending on the modules loaded at this point the new PCI devices may be > > - bound to nfp_netvf driver. > > + -k option shows the device driver, if any, that the devices are bound > > to. > > + Depending on the modules loaded, at this point the new PCI devices may > > be > > + bound to the ``nfp`` kernel driver or ``vfio-pci``. > > > > > > Flow offload > > @@ -193,13 +205,13 @@ The flower firmware application requires the PMD > > running two services: > > > > * PF vNIC service: handling the feedback traffic. > > * ctrl vNIC service: communicate between PMD and firmware through > > - control message. > > + control messages. > > > > To achieve the offload of flow, the representor ports are exposed to OVS. > > -The flower firmware application support representor port for VF and > > physical > > -port. There will always exist a representor port for each physical port, > > -and the number of the representor port for VF is specified by the user > > through > > -parameter. > > +The flower firmware application supports VF, PF, and physical port > > representor > > +ports. > > Again old document and new one is not saying same thing, is it intentional? > > Old one says: "Having representor ports for both VF and PF is supported." > > New one says: "FW supports representor port, VF and PF." > > > There will always exist a representor port for a PF and each physical > > +port. The number of the representor ports for VFs are specified by the user > > +through a parameter. > > > > In the Rx direction, the flower firmware application will prepend the input > > port information into metadata for each packet which can't offloaded. The > > PF > > @@ -207,12 +219,12 @@ vNIC service will keep polling packets from the > > firmware, and multiplex them > > to the corresponding representor port. > > > > In the Tx direction, the representor port will prepend the output port > > -information into metadata for each packet, and then send it to firmware > > through > > -PF vNIC. > > +information into metadata for each packet, and then send it to the firmware > > +through the PF vNIC. > > > > -The ctrl vNIC service handling various control message, like the creation > > and > > -configuration of representor port, the pattern and action of flow rules, > > the > > -statistics of flow rules, and so on. > > +The ctrl vNIC service handles various control messages, for example, the > > +creation and configuration of a representor port, the pattern and action of > > +flow rules, the statistics of flow rules, etc. > > > > Metadata Format > > --------------- > -- Kind Regards, Niklas Söderlund
