On Thu, 2019-03-07 at 14:24 +0100, Rastislav Cernay wrote: > From: Rastislav Cernay <cer...@netcope.com> > > Note: New netcope-commong pcg on which is support for pkg-config, > will be released soon and uploaded on official site. > > Added new net driver for Netcope nfb cards > > Signed-off-by: Rastislav Cernay <cer...@netcope.com> > --- > v2: remove unnecessary cast > remove unnecessary zeroing > move declaration to not mix with code > restore skeleton example > v3: add release notes > add doc to doc index > add architecture limits to doc > edit features list > add .map file > change link to dependecies to official vendor site > move declarations out of code > remove false comments (rte_errno is set) > comments to c89 style > remove log from main rx loop > remove redundant code > v4: add random initial MAC > fix format > move declarations out of code > fix error log in TX > q_errors is for reception errors > pmd is experimental > dependency check fixed > support for pkg-config > MAINTAINERS | 7 + > config/common_base | 4 + > devtools/test-build.sh | 1 + > doc/guides/nics/features/nfb.ini | 18 + > doc/guides/nics/index.rst | 1 + > doc/guides/nics/nfb.rst | 143 +++++++ > doc/guides/rel_notes/release_19_02.rst | 5 + > drivers/net/Makefile | 1 + > drivers/net/meson.build | 1 + > drivers/net/nfb/Makefile | 44 +++ > drivers/net/nfb/meson.build | 15 + > drivers/net/nfb/nfb.h | 50 +++ > drivers/net/nfb/nfb_ethdev.c | 647 > ++++++++++++++++++++++++++++++++ > drivers/net/nfb/nfb_rx.c | 127 +++++++ > drivers/net/nfb/nfb_rx.h | 231 ++++++++++++ > drivers/net/nfb/nfb_rxmode.c | 100 +++++ > drivers/net/nfb/nfb_rxmode.h | 96 +++++ > drivers/net/nfb/nfb_stats.c | 77 ++++ > drivers/net/nfb/nfb_stats.h | 56 +++ > drivers/net/nfb/nfb_tx.c | 113 ++++++ > drivers/net/nfb/nfb_tx.h | 222 +++++++++++ > drivers/net/nfb/rte_nfb_pmd_version.map | 4 + > mk/rte.app.mk | 2 + > 23 files changed, 1965 insertions(+) > create mode 100644 doc/guides/nics/features/nfb.ini > create mode 100644 doc/guides/nics/nfb.rst > create mode 100644 drivers/net/nfb/Makefile > create mode 100644 drivers/net/nfb/meson.build > create mode 100644 drivers/net/nfb/nfb.h > create mode 100644 drivers/net/nfb/nfb_ethdev.c > create mode 100644 drivers/net/nfb/nfb_rx.c > create mode 100644 drivers/net/nfb/nfb_rx.h > create mode 100644 drivers/net/nfb/nfb_rxmode.c > create mode 100644 drivers/net/nfb/nfb_rxmode.h > create mode 100644 drivers/net/nfb/nfb_stats.c > create mode 100644 drivers/net/nfb/nfb_stats.h > create mode 100644 drivers/net/nfb/nfb_tx.c > create mode 100644 drivers/net/nfb/nfb_tx.h > create mode 100644 drivers/net/nfb/rte_nfb_pmd_version.map
<...> > + > +.. note:: > + > + This driver has external dependencies. > + Therefore it is disabled in default configuration files. > + It can be enabled by setting ``CONFIG_RTE_LIBRTE_PMD_NFB=y`` > + and recompiling. > + > +.. note:: > + > + Currently the driver is supported only on x86_64 architectures. > + Only x86_64 versions of the external libraries are provided. > + > +Prerequisites > +------------- > + > +This PMD requires kernel modules which are responsible for > initialization and > +allocation of resources needed for nfb layer function. > +Communication between PMD and kernel modules is mediated by libnfb > library. > +These kernel modules and library are not part of DPDK and must be > installed > +separately: > + > +* **libnfb library** > + > + The library provides API for initialization of nfb transfers, > receiving and > + transmitting data segments. > + > +* **Kernel modules** > + > + * nfb > + > + Kernel modules manage initialization of hardware, allocation and > + sharing of resources for user space applications. > + > +Dependencies can be found here: > +`Netcope common < > https://www.netcope.com/en/company/community-support/dpdk-libsze2#NFB>`_ > . I see netcope-common provides a pkg-config file, that's great. However it's a bit broken at the moment: Name: netcope-common Version: 6.3.0 Description: Common software package for NetCOPE platform Libs: -L${libdir} Cflags: -I${includedir} There is no linker flag, and Libs.private is missing. It should have: Libs: -L${libdir} -lnfb Libs.private: -lfdt Could you please get it fixed? > +Versions of the packages > +~~~~~~~~~~~~~~~~~~~~~~~~ > + > +The minimum version of the provided packages: > + > +* for DPDK from 19.02 > + > +Configuration > +------------- > + > +These configuration options can be modified before compilation in > the > +``.config`` file: > + > +* ``CONFIG_RTE_LIBRTE_PMD_NFB`` default value: **n** > + > + Value **y** enables compilation of nfb PMD. > + > +Using the NFB PMD > +---------------------- > + > +Kernel modules have to be loaded before running the DPDK > application. > + > +NFB card architecture > +--------------------- > + > +The NFB cards are multi-port multi-queue cards, where (generally) > data from any > +Ethernet port may be sent to any queue. > +They are represented in DPDK as a single port. > + > +NFB-200G2QL card employs an addon cable which allows to connect it > to two > +physical PCI-E slots at the same time (see the diagram below). > +This is done to allow 200 Gbps of traffic to be transferred through > the PCI-E > +bus (note that a single PCI-E 3.0 x16 slot provides only 125 Gbps > theoretical > +throughput). > + > +Although each slot may be connected to a different CPU and therefore > to a different > +NUMA node, the card is represented as a single port in DPDK. To work > with data > +from the individual queues on the right NUMA node, connection of > NUMA nodes on > +first and last queue (each NUMA node has half of the queues) need to > be checked. > + > +.. figure:: img/szedata2_nfb200g_architecture.svg > + :align: center > + > + NFB-200G2QL high-level diagram > + > +Limitations > +----------- > + > +Driver is usable only on Linux architecture. > + > +Since a card is always represented as a single port, but can be > connected to two > +NUMA nodes, there is need for manual check where master/slave is > connected. > + > +Example of usage > +---------------- > + > +Read packets from 0. and 1. receive queue and write them to 0. and > 1. > +transmit queue: > + > +.. code-block:: console > + > + $RTE_TARGET/app/testpmd -l 0-3 -n 2 \ > + -- --port-topology=chained --rxq=2 --txq=2 --nb-cores=2 -i -a > + > +Example output: > + > +.. code-block:: console > + > + [...] > + EAL: PCI device 0000:06:00.0 on NUMA socket -1 > + EAL: probe driver: 1b26:c1c1 net_nfb > + PMD: Initializing NFB device (0000:06:00.0) > + PMD: Available DMA queues RX: 8 TX: 8 > + PMD: NFB device (0000:06:00.0) successfully initialized > + Interactive-mode selected > + Auto-start selected > + Configuring Port 0 (socket 0) > + Port 0: 00:11:17:00:00:00 > + Checking link statuses... > + Port 0 Link Up - speed 10000 Mbps - full-duplex > + Done > + Start automatic packet forwarding > + io packet forwarding - CRC stripping disabled - > packets/burst=32 > + nb forwarding cores=2 - nb forwarding ports=1 > + RX queues=2 - RX desc=128 - RX free threshold=0 > + RX threshold registers: pthresh=0 hthresh=0 wthresh=0 > + TX queues=2 - TX desc=512 - TX free threshold=0 > + TX threshold registers: pthresh=0 hthresh=0 wthresh=0 > + TX RS bit threshold=0 - TXQ flags=0x0 > + testpmd> > diff --git a/doc/guides/rel_notes/release_19_02.rst > b/doc/guides/rel_notes/release_19_02.rst > index 3372c4d..9955b6b 100644 > --- a/doc/guides/rel_notes/release_19_02.rst > +++ b/doc/guides/rel_notes/release_19_02.rst > @@ -54,6 +54,11 @@ New Features > Also, make sure to start the actual text at the margin. > ========================================================= > > +* **Added an experimental net PMD NFB.** > + > + Added the new ``nfb`` net driver for Netcope NFB cards. See > + the :doc:`../nics/nfb` NIC guide for more details on this new > driver. > + > * **Added support to free hugepages exactly as originally > allocated.** > > Some applications using memory event callbacks (especially for > managing > diff --git a/drivers/net/Makefile b/drivers/net/Makefile > index 670d7f7..013c551 100644 > --- a/drivers/net/Makefile > +++ b/drivers/net/Makefile > @@ -38,6 +38,7 @@ DIRS-$(CONFIG_RTE_LIBRTE_MLX5_PMD) += mlx5 > DIRS-$(CONFIG_RTE_LIBRTE_MVNETA_PMD) += mvneta > DIRS-$(CONFIG_RTE_LIBRTE_MVPP2_PMD) += mvpp2 > DIRS-$(CONFIG_RTE_LIBRTE_NETVSC_PMD) += netvsc > +DIRS-$(CONFIG_RTE_LIBRTE_PMD_NFB) += nfb > DIRS-$(CONFIG_RTE_LIBRTE_NFP_PMD) += nfp > DIRS-$(CONFIG_RTE_LIBRTE_BNXT_PMD) += bnxt > DIRS-$(CONFIG_RTE_LIBRTE_PMD_NULL) += null > diff --git a/drivers/net/meson.build b/drivers/net/meson.build > index 45da3bb..6733a19 100644 > --- a/drivers/net/meson.build > +++ b/drivers/net/meson.build > @@ -27,6 +27,7 @@ drivers = ['af_packet', > 'mvneta', > 'mvpp2', > 'netvsc', > + 'nfb', > 'nfp', > 'null', 'octeontx', 'pcap', 'qede', 'ring', > 'sfc', > diff --git a/drivers/net/nfb/Makefile b/drivers/net/nfb/Makefile > new file mode 100644 > index 0000000..968184e > --- /dev/null > +++ b/drivers/net/nfb/Makefile > @@ -0,0 +1,44 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Cesnet > +# Copyright(c) 2018 Netcope Technologies, a.s. <i...@netcope.com> > +# All rights reserved. > + > +include $(RTE_SDK)/mk/rte.vars.mk > + > +# > +# library name > +# > +LIB = librte_pmd_nfb.a > + > +INCLUDES :=-I$(SRCDIR) > + > + > +CFLAGS += -O3 > +CFLAGS += $(WERROR_FLAGS) > +CFLAGS += -DALLOW_EXPERIMENTAL_API > +CFLAGS += $(shell command -v pkg-config > /dev/null 2>&1 && pkg- > config --cflags libnfb) This will not work - the pkg-config filename is netcope-common.pc, not libnfb.pc. The convention would be to rename the pkg-config file to libnfb.pc, but if you want to keep the current name then all pkg-config calls in this patch need to be updated. > +LDLIBS += -lnfb Do not hard-code this, pkg-config is used below. > +LDLIBS += -lrte_eal -lrte_mbuf -lrte_mempool -lrte_ring > +LDLIBS += -lrte_ethdev -lrte_net -lrte_kvargs > +LDLIBS += -lrte_bus_pci > +LDLIBS += $(shell command -v pkg-config > /dev/null 2>&1 && pkg- > config --libs libnfb) Use: LDLIBS += $(shell command -v pkg-config > /dev/null 2>&1 && pkg-config --libs libnfb || echo "-lnfb") This way you have a fallback if pkg-config is not installed. > +EXPORT_MAP := rte_pmd_nfb_version.map > + > +LIBABIVER := 1 > + > +# > +# all source are stored in SRCS-y > +# > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) += nfb_ethdev.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) += nfb_rx.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) += nfb_tx.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) += nfb_stats.c > +SRCS-$(CONFIG_RTE_LIBRTE_PMD_NFB) += nfb_rxmode.c > + > +# > +# Export include files > +# > +SYMLINK-y-include += > + > +include $(RTE_SDK)/mk/rte.lib.mk > diff --git a/drivers/net/nfb/meson.build > b/drivers/net/nfb/meson.build > new file mode 100644 > index 0000000..16df146 > --- /dev/null > +++ b/drivers/net/nfb/meson.build > @@ -0,0 +1,15 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright(c) 2018 Cesnet > +# Copyright(c) 2018 Netcope Technologies, a.s. <i...@netcope.com> > +# All rights reserved. > + > +dep = cc.find_library('nfb', required: false) > + > +build = dep.found() and cc.has_header('nfb/nfb.h', dependencies: > dep) > + > +allow_experimental_apis = true > + > +ext_deps += dep > + > +sources = files('nfb_rx.c', 'nfb_tx.c', 'nfb_stats.c', > 'nfb_ethdev.c', 'nfb_rxmode.c') > +pkgconfig_extra_libs += '-lnfb' libnfb ships a pkg-config file, so please do this instead so that static compilation can work correctly: dep = dependency('netcope-common', required: false) build = dep.found() allow_experimental_apis = true ext_deps += dep sources = files('nfb_rx.c', 'nfb_tx.c', 'nfb_stats.c', 'nfb_ethdev.c', 'nfb_rxmode.c') -- Kind regards, Luca Boccassi
