On Mon, Nov 2, 2020 at 4:58 AM Timothy McDaniel
<timothy.mcdan...@intel.com> wrote:
>
> The following patch series adds support for a new eventdev PMD. The DLB
> PMD adds support for the Intel Dynamic Load Balancer (DLB) hardware.
> The DLB is a PCIe device that provides load-balanced, prioritized
> scheduling of core-to-core communication. The device consists of
> queues and arbiters that connect producer and consumer cores, and
> implements load-balanced queueing features including:
> - Lock-free multi-producer/multi-consumer operation.
> - Multiple priority levels for varying traffic types.
> - 'Direct' traffic (i.e. multi-producer/single-consumer)
> - Simple unordered load-balanced distribution.
> - Atomic lock-free load balancing across multiple consumers.
> - Queue element reordering feature allowing ordered load-balanced
> distribution.
>
> The DLB hardware supports both load balanced and directed ports and
> queues. Unlike other eventdev devices already in the repo, not all
> DLB ports and queues are equally capable. In particular, directed
> ports are limited to a single link, and must be connected to a directed
> queue.
> Additionally, even though LDB ports may link multiple queues, the
> number of queues that may be linked is limited by hardware. Another
> difference is that DLB does not have a straightforward way of carrying
> the flow_id in the queue elements (QE) that the hardware operates on.
>
> While reviewing the code, please be aware that this PMD has full
> control over the DLB hardware. Intel will be extending the DLB PMD
> in the future (not as part of this first series) with a mode that we
> refer to as the bifurcated PMD. The bifurcated PMD communicates with a
> kernel driver to configure the device, ports, and queues, and memory
> maps device MMIO so datapath operations occur purely in user-space.
>
> The framework to support both the PF PMD and bifurcated PMD exists in
> this patchset, and is why the iface.[ch] layer is present.
Series applied to dpdk-next-eventdev/for-main with the following fix. Thanks.
diff --git a/doc/guides/eventdevs/dlb.rst b/doc/guides/eventdevs/dlb.rst
index d44afcdcf..4c4f56b2b 100644
--- a/doc/guides/eventdevs/dlb.rst
+++ b/doc/guides/eventdevs/dlb.rst
@@ -2,7 +2,7 @@
Copyright(c) 2020 Intel Corporation.
Driver for the Intel® Dynamic Load Balancer (DLB)
+=================================================
-==================================================
The DPDK dlb poll mode driver supports the Intel® Dynamic Load Balancer.
>
> Major changes in V16
> ====================
> Address additional comments from David Marchand:
> - converted printfs in dlb/pf/dlb_main.c to DLB_LOG
> - fixed a repeated word error in dlb/pf/base/osdep_bitmap.h
> - caught up with marking the patches that Gage reviewed
>
> Major changes in V15
> ====================
> Address comments from David Marchand:
> - this patch-set is based on Nov 1, 2020 dpdk-next-eventdev
> - fix docs build (doxy-api.conf.in and doxy-api-index.md)
> - restore blank line in MAINTAINERS file
> - move dlb announcement in release_20_11.rst after ethdev
> - use headers = files() for exported meson public headers
> - fix a typo in 'add documentation ..." commit message
> - use eal version of cldemote
> - convert a couple of printfs to LOG messages
> - fix missing "~" in dlb documentation
> - delay introduction of _delayed token pop functions to
> token pop commit (fixes 8 or so unused function errors)
> - all patches build incrementally (gcc), and checkpatches reports
> success
> - I am not able to run clang locally. If clang errors are still
> present I will ask IT to install clang on a build server tomorrow.
>
> Major changes in V14
> ====================
> - Fixed format errors in doc/api/doxy-api-index.md
> - Delayed introduction of dlb2_consume_qe_immediate until
> add-dequeue-and-its-burst-variants.patch
> - Delayed introduction of dlb2_construct_token_pop_qe until
> add-PMD-s-token-pop-public-interface.patch
> - Delayed introduction of dlb_equeue_*_delayed until
> add dequeue and its burst variants.patch
>
> Major changes in V13
> ====================
> - removed now unused functions dlb_umwait and dlb_umonitor
>
> Major changes in V12
> ====================
> - Fix CENTOS build error: use __m128i instead of __v2di with
> _mm_stream_si128
>
> Major changes in V11
> ====================
> - removed unused function, fixing build error
> - fixed typo in port_setup commit message
> - this patch series is based on dpdk-next-eventdev
>
> Major changes in v10
> =====================
> - convert to use rte_power_monitor patches
> - replace __builtin_ia32_movntdq() with _mm_stream_si128()
> - remove unused functions in dlb_selftest.c
>
> Major changes in v9
> =====================
> - fixed a build error due to __rte_cache_aligned being placed after
> the ";" character, instead of before it.
>
> Major changes in v8 after dpdk reviews
> =====================
> - moved introduction of dlb in relnotes_20_11 to first patch in series
> - fixed underlines in dlb.rst that were too short
> - note that the code still uses its private byte-encoded versions of
> umonitor/umwait, rather than the new functions in the power
> patch that are built on top of those intrinsics. This is intentional.
>
> Major changes in v7 after dpdk reviews
> =====================
> - updated MAINTAINERS file to alphabetically insert DLB
> - don't create RTE_ symbols in PMD
> - converted to use version.map scheme
> - converted to use .._master_lcore instead of .._main_lcore
> - this patch set is based on dpdk-next-eventdev
>
> Major changes in v6 after dpdk reviews:
> =====================
> - fixed meson conditional build. Moved test into driver’s meson.build
> file instead of event/meson.build
> - documentation is populated as associated code is introduced
> - add log_register in add dynamic logging patch
> - rename RTE_xxx symbol(s) as DLB2_xxx
> - replaced function ptr enqueue_four with direct call to movdir64b
> - remove unused port_pages
> - broke up probe patch into 3 smaller patches for easier review
> - changed param order of movdir64b/movntdq to match intrinsics
> - added self to MAINTAINERS files
> - squashed announcement of availability into last patch in series
> - correct spelling errors and delete repeated words
> - DPDK_21.0 -> DPDK 21 in map file
> - add experimental banner to public structs and APIs
> - implemented other suggestions from code reviews of DLB2 PMD. The
> software is very similar in form so some DLB2 reviews comments
> were applicable to DLB as well
>
> Major changes in v5 after dpdk reviews and additional internal reviews
> by colleagues at Intel:
> ================
> - implement changes requested in code reviews by Gage Eads and Mike Chen
> - fix a memzone leak
> - convert to use eal rte-cpuflags patch from Liang Ma
>
> Major changes in v4 after dpdk reviews and additional internal reviews
> by colleagues at Intel:
> ================
> - Remove make infrastructure
> - shared code (pf/base) is now added incrementally
> - flexible interface (iface.[ch]) is now added incrementally
> - removed calls to rte_panic
> - do not call pthread_create directly
> - remove unused internal API, os_time
> - convert rte_atomic to __atomic builtins
> - broke out eventdev ABI changes, test/api changes, and new internal PCI
> named probe API
> - relocated enqueue logic to enqueue patch
>
> Major Changes in V3:
> ================
> - Fixed a memory corruption issue due to not allocating enough CQ
> memory for depths < 8. Hardware requires minimum allocation to be
> at least 8 entries.
> - Address review comments from Gage and Mattias.
> - Remove versioning
> - minor formatting changes
>
> Major changes in V2:
> ================
> - Correct ABI break that was present in V1.
> - Address some of the review comments received from Mattias.
> I will address the remaining items identified by Mattias in the next
> patch delivery.
> - General code cleanup based on internal code reviews
>
> Depends-on: patch-82202 ("eventdev: increase MAX QUEUES PER DEV to 255")
>
> Timothy McDaniel (23):
> event/dlb: add documentation and meson infrastructure
> event/dlb: add dynamic logging
> event/dlb: add private data structures and constants
> event/dlb: add definitions shared with LKM or shared code
> event/dlb: add inline functions
> event/dlb: add eventdev probe
> event/dlb: add flexible interface
> event/dlb: add probe-time hardware init
> event/dlb: add xstats
> event/dlb: add infos get and configure
> event/dlb: add queue and port default conf
> event/dlb: add queue setup
> event/dlb: add port setup
> event/dlb: add port link
> event/dlb: add port unlink and port unlinks in progress
> event/dlb: add eventdev start
> event/dlb: add enqueue and its burst variants
> event/dlb: add dequeue and its burst variants
> event/dlb: add eventdev stop and close
> event/dlb: add PMD's token pop public interface
> event/dlb: add PMD self-tests
> event/dlb: add queue and port release
> event/dlb: add timeout ticks entry point
>
> MAINTAINERS | 5 +
> app/test/test_eventdev.c | 7 +
> config/rte_config.h | 6 +
> doc/api/doxy-api-index.md | 3 +-
> doc/api/doxy-api.conf.in | 1 +
> doc/guides/eventdevs/dlb.rst | 341 ++
> doc/guides/eventdevs/index.rst | 1 +
> doc/guides/rel_notes/release_20_11.rst | 5 +
> drivers/event/dlb/dlb.c | 4079 +++++++++++++++
> drivers/event/dlb/dlb_iface.c | 79 +
> drivers/event/dlb/dlb_iface.h | 82 +
> drivers/event/dlb/dlb_inline_fns.h | 36 +
> drivers/event/dlb/dlb_log.h | 25 +
> drivers/event/dlb/dlb_priv.h | 513 ++
> drivers/event/dlb/dlb_selftest.c | 1539 ++++++
> drivers/event/dlb/dlb_user.h | 814 +++
> drivers/event/dlb/dlb_xstats.c | 1217 +++++
> drivers/event/dlb/meson.build | 22 +
> drivers/event/dlb/pf/base/dlb_hw_types.h | 334 ++
> drivers/event/dlb/pf/base/dlb_osdep.h | 310 ++
> drivers/event/dlb/pf/base/dlb_osdep_bitmap.h | 441 ++
> drivers/event/dlb/pf/base/dlb_osdep_list.h | 131 +
> drivers/event/dlb/pf/base/dlb_osdep_types.h | 31 +
> drivers/event/dlb/pf/base/dlb_regs.h | 2368 +++++++++
> drivers/event/dlb/pf/base/dlb_resource.c | 6904
> ++++++++++++++++++++++++++
> drivers/event/dlb/pf/base/dlb_resource.h | 876 ++++
> drivers/event/dlb/pf/dlb_main.c | 586 +++
> drivers/event/dlb/pf/dlb_main.h | 47 +
> drivers/event/dlb/pf/dlb_pf.c | 750 +++
> drivers/event/dlb/rte_pmd_dlb.c | 38 +
> drivers/event/dlb/rte_pmd_dlb.h | 77 +
> drivers/event/dlb/version.map | 9 +
> drivers/event/meson.build | 2 +-
> 33 files changed, 21677 insertions(+), 2 deletions(-)
> create mode 100644 doc/guides/eventdevs/dlb.rst
> create mode 100644 drivers/event/dlb/dlb.c
> create mode 100644 drivers/event/dlb/dlb_iface.c
> create mode 100644 drivers/event/dlb/dlb_iface.h
> create mode 100644 drivers/event/dlb/dlb_inline_fns.h
> create mode 100644 drivers/event/dlb/dlb_log.h
> create mode 100644 drivers/event/dlb/dlb_priv.h
> create mode 100644 drivers/event/dlb/dlb_selftest.c
> create mode 100644 drivers/event/dlb/dlb_user.h
> create mode 100644 drivers/event/dlb/dlb_xstats.c
> create mode 100644 drivers/event/dlb/meson.build
> create mode 100644 drivers/event/dlb/pf/base/dlb_hw_types.h
> create mode 100644 drivers/event/dlb/pf/base/dlb_osdep.h
> create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_bitmap.h
> create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_list.h
> create mode 100644 drivers/event/dlb/pf/base/dlb_osdep_types.h
> create mode 100644 drivers/event/dlb/pf/base/dlb_regs.h
> create mode 100644 drivers/event/dlb/pf/base/dlb_resource.c
> create mode 100644 drivers/event/dlb/pf/base/dlb_resource.h
> create mode 100644 drivers/event/dlb/pf/dlb_main.c
> create mode 100644 drivers/event/dlb/pf/dlb_main.h
> create mode 100644 drivers/event/dlb/pf/dlb_pf.c
> create mode 100644 drivers/event/dlb/rte_pmd_dlb.c
> create mode 100644 drivers/event/dlb/rte_pmd_dlb.h
> create mode 100644 drivers/event/dlb/version.map
>
> --
> 2.6.4
>
