Re: [PATCH v8 5/5] dts: add API doc generation

[Juraj Linkeš]

On 2. 8. 2024 15:53, Thomas Monjalon wrote:

02é x/08/2024 12:48, Juraj Linkeš:

On 1. 8. 2024 17:07, Thomas Monjalon wrote:

01/08/2024 15:03, Juraj Linkeš:

On 30. 7. 2024 15:51, Thomas Monjalon wrote:

12/07/2024 10:57, Juraj Linkeš:

+dts_root = environ.get('DTS_ROOT')


Why does it need to be passed as an environment variable?
Isn't it a fixed absolute path?


The path to DTS needs to be passed in some way (and added to sys.path)
so that Sphinx knows where the sources are in order to import them.

Do you want us to not pass the path, but just hardcode it here? I didn't
really think about that, maybe that could work.


I think hardcode is better here. xFalse,

  'navigation_depth': -1,
  }

The sidebar configuration is conditional, so we have to pass something
to indicate dts build. I'll change it so that we look for 'dts' in src
in call-sphinx-build.py (we're in the dts doc directory, indicating dts
build) and set the DTS_BUILD env var which we can use in conf.py. I
didn't find a better way to do this as conf.py doesn't have any
information about the build itself (and no path that conf.py has access
to points to anything dts). Here's how it'll look:

if environ.get('DTS_BUILD'):
  path.append(path_join(dirname(dirname(dirname(__file__))), 'dts'))
  # DTS Sidebar config.
  html_theme_options = {
  'collapse_navigation': False,
  'navigation_depth': -1,  # unlimited depth
  }


OK



+To build DTS API docs, install the dependencies with Poetry, then enter its 
shell:


I don't plan to use Poetry on my machine.
Can we simply describe the dependencies even if the versions are not specified?


The reason we don't list the dependencies anywhere is that doing it with
Poetry is much easier (and a bit safer, as Poetry is going to install
tested versions).

But I can add references to the two relevant sections of
dts/pyproject.toml which contain the dependencies with a note that they
can be installed with pip (and I guess that would be another
dependency), but at that point it's that not much different than using
Poetry.


I want to use my system package manager.
I am from this old school thinking we should have a single package manager in a 
system.



I understand and would also prefer that, but it just doesn't work for
Python. Not all packages are available from the package managers, and
Python projects should not use system packages as there are frequently
version mismatches between the system packages and what the project
needs (the APIs could be different as well as behavior; a problem we've
seen with Scapy). Poetry is one of the tools that tries to solve this
well-known Python limitation.


I fully agree for DTS runtime.
I'm expecting the dependencies are more tolerant for DTS doc.


I've done a quick search of what's available in Ubuntu and two packages
aren't available, types-PyYAML (which maybe we could do without, I'll
have to test) and aenum (which is currently needed for the capabilities
patch; if absolutely necessary, maybe I could find a solution without
aenum). But even with this we can't be sure that the system package
versions will work.


We need them all to generate the documentation?



We actually may not. The Python docstrings are part of the code (stored 
in the __docstring__ attribute of everything), Sphinx (more precisely 
the autodoc extension [0]) imports all the code to access the docstrings 
and to do that, it needs the dependencies.


However, I found a config option that mocks imports from the specified 
modules [1], so what we can do is list the missing modules there (and we 
can build without the dependencies). If we do this, we could emit a 
warning from Sphinx, although the resulting docs don't seem any 
different according to the basic tests I did.


So this means we can do a build with both passing the target and passing 
-Denable_docs, provided I won't encounter anything wild. I'll change the 
docs accordingly.


[0] 
https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#ensuring-the-code-can-be-imported
[1] 
https://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_mock_imports



+.. code-block:: console
+
+   poetry install --no-root --with docs
+   poetry shell
+
+The documentation is built using the standard DPDK build system.
+After executing the meson command and entering Poetry's shell, build the 
documentation with:
+
+.. code-block:: console
+
+   ninja -C build dts-doc


Don't we rely on the Meson option "enable_docs"?


I had a discussion about this with Bruce, but I can't find it anywhere,
so here's what I remember:
1. We didn't want to tie the dts api doc build to dpdk doc build because
of the dependencies.


Sure
But we could just skip if dependencies are not met?


Maybe we could add a script that would check the dependencies. I'll see
what I can do.


OK thanks

[PATCH] net/virtio-user: reset used index counter in dev reset

[Kommula Shiva Shankar]

When the virtio device is reinitialized during ethdev reconfiguration,
all the virtio rings are recreated and repopulated on the device.
Accordingly, reset the used index counter value back to zero.

Signed-off-by: Kommula Shiva Shankar 
---
 drivers/net/virtio/virtio_user_ethdev.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/drivers/net/virtio/virtio_user_ethdev.c 
b/drivers/net/virtio/virtio_user_ethdev.c
index ae6593ba0b..d60c7e188c 100644
--- a/drivers/net/virtio/virtio_user_ethdev.c
+++ b/drivers/net/virtio/virtio_user_ethdev.c
@@ -204,6 +204,7 @@ virtio_user_setup_queue_packed(struct virtqueue *vq,
vring->device = (void *)(uintptr_t)used_addr;
dev->packed_queues[queue_idx].avail_wrap_counter = true;
dev->packed_queues[queue_idx].used_wrap_counter = true;
+   dev->packed_queues[queue_idx].used_idx = 0;
 
for (i = 0; i < vring->num; i++)
vring->desc[i].flags = 0;
-- 
2.43.0

Re: 22.11.6 patches review and test

[Luca Boccassi]

On Mon, 5 Aug 2024 at 06:45, YangHang Liu  wrote:
>
> RedHat QE tested below 18 scenarios on RHEL 9.2 and didn't find any new dpdk 
> issues.
>
> VM with device assignment(PF) throughput testing(1G hugepage size): PASS
> VM with device assignment(PF) throughput testing(2M hugepage size) : PASS
> VM with device assignment(VF) throughput testing: PASS
> PVP (host dpdk testpmd as vswitch) 1Q: throughput testing: PASS
> PVP vhost-user 2Q throughput testing: PASS
> PVP vhost-user 1Q - cross numa node throughput testing: PASS
> VM with vhost-user 2 queues throughput testing: PASS
> vhost-user reconnect with dpdk-client, qemu-server qemu reconnect: PASS
> vhost-user reconnect with dpdk-client, qemu-server ovs reconnect: PASS
> PVP  reconnect with dpdk-client, qemu-server: PASS
> PVP 1Q live migration testing: PASS
> PVP 1Q cross numa node live migration testing: PASS
> VM with ovs+dpdk+vhost-user 1Q live migration testing: PASS
> VM with ovs+dpdk+vhost-user 1Q live migration testing (2M): PASS
> VM with ovs+dpdk+vhost-user 2Q live migration testing: PASS
> VM with ovs+dpdk+vhost-user 4Q live migration testing: PASS
> Host PF + DPDK testing: PASS
> Host VF + DPDK testing: PASS
>
> Test Versions:
>
> qemu-kvm-7.2.0
> kernel 5.14
> libvirt 9.0
> openvswitch 3.1
> git log
>
> commit 2480dbd434234a40e7f999ced4650581fd64a24e
>
> Author: Luca Boccassi 
>
> Date: Wed Jul 31 20:35:00 2024 +0100
>
> version: 22.11.6-rc1
>
> Signed-off-by: Luca Boccassi 
>
> Test device : X540-AT2 NIC(ixgbe, 10G)
>
> Tested-by: Yanghang Liu

Thank you!

Re: [PATCH v2 1/1] dts: add text parser for testpmd verbose output

[Nicholas Pratte]

> I like the sound of this idea a lot actually since it would remove the
> chance of the output just completely being thrown away. In my own test
> suite I managed to dance around this by strategically placing my
> testpmd commands, but this could save people some headache in the
> future. I feel like this wouldn't be something overly complicated to
> implement either, all we would have to do is extend the send_command
> method in the TestpmdShell class and check a boolean for if verbose is
> on, extract this output. If/how to clear this list would be something
> to think about, but I would say that, in general, the idea of making
> sure we don't lose information is something that I'm all for.

That's a good point that you could just modify the send_command
method. In my head I was thinking that we'd have to modify each
individual method! Totally forget that all those testpmd methods I was
thinking about stem from send_command().

[v1,1/5] raw/zxdh: introduce zxdh raw device driver

[Yong Zhang]

Introduce rawdev driver support for ZXDH which
can help to connect two separate hosts with each other.

Signed-off-by: Yong Zhang 
---
 MAINTAINERS|   5 +
 doc/guides/rawdevs/index.rst   |   1 +
 doc/guides/rawdevs/zxdh.rst|  29 +
 drivers/raw/meson.build|   1 +
 drivers/raw/zxdh/meson.build   |   5 +
 drivers/raw/zxdh/zxdh_rawdev.c | 221 +
 drivers/raw/zxdh/zxdh_rawdev.h | 118 ++
 7 files changed, 380 insertions(+)
 create mode 100644 doc/guides/rawdevs/zxdh.rst
 create mode 100644 drivers/raw/zxdh/meson.build
 create mode 100644 drivers/raw/zxdh/zxdh_rawdev.c
 create mode 100644 drivers/raw/zxdh/zxdh_rawdev.h

diff --git a/MAINTAINERS b/MAINTAINERS
index c71ca2a28e..867a2f9b1c 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1500,6 +1500,11 @@ M: Gagandeep Singh 
 F: drivers/raw/dpaa2_cmdif/
 F: doc/guides/rawdevs/dpaa2_cmdif.rst
 
+ZXDH
+M: Yong Zhang 
+F: drivers/raw/zxdh/
+F: doc/guides/rawdevs/zxdh.rst
+
 
 Packet processing
 -
diff --git a/doc/guides/rawdevs/index.rst b/doc/guides/rawdevs/index.rst
index f34315f051..d85a4b7148 100644
--- a/doc/guides/rawdevs/index.rst
+++ b/doc/guides/rawdevs/index.rst
@@ -16,3 +16,4 @@ application through rawdev API.
 dpaa2_cmdif
 ifpga
 ntb
+zxdh
diff --git a/doc/guides/rawdevs/zxdh.rst b/doc/guides/rawdevs/zxdh.rst
new file mode 100644
index 00..a75fd8a748
--- /dev/null
+++ b/doc/guides/rawdevs/zxdh.rst
@@ -0,0 +1,29 @@
+..  SPDX-License-Identifier: BSD-3-Clause
+Copyright 2024 ZTE Corporation
+
+ZXDH Rawdev Driver
+==
+
+The ``zxdh`` rawdev driver is an implementation of the rawdev API,
+that provides communication between two separate hosts.
+This is achieved via using the GDMA controller of Dinghai SoC,
+which can be configured through exposed MPF devices.
+
+Device Setup
+-
+
+It is recommended to bind the ZXDH MPF kernel driver for MPF devices (Not 
mandatory).
+The kernel drivers can be downloaded at `ZTE Official Website
+`_.
+
+Initialization
+--
+
+The ``zxdh`` rawdev driver needs to work in IOVA PA mode.
+Consider using ``--iova-mode=pa`` in the EAL options.
+
+Platform Requirement
+
+
+This PMD is only supported on ZTE Neo Platforms:
+- Neo X510/X512
diff --git a/drivers/raw/meson.build b/drivers/raw/meson.build
index 05cad143fe..237d1bdd80 100644
--- a/drivers/raw/meson.build
+++ b/drivers/raw/meson.build
@@ -12,5 +12,6 @@ drivers = [
 'ifpga',
 'ntb',
 'skeleton',
+'zxdh',
 ]
 std_deps = ['rawdev']
diff --git a/drivers/raw/zxdh/meson.build b/drivers/raw/zxdh/meson.build
new file mode 100644
index 00..266d3db6d8
--- /dev/null
+++ b/drivers/raw/zxdh/meson.build
@@ -0,0 +1,5 @@
+#SPDX-License-Identifier: BSD-3-Clause
+#Copyright 2024 ZTE Corporation
+
+deps += ['rawdev', 'kvargs', 'mbuf', 'bus_pci']
+sources = files('zxdh_rawdev.c')
diff --git a/drivers/raw/zxdh/zxdh_rawdev.c b/drivers/raw/zxdh/zxdh_rawdev.c
new file mode 100644
index 00..43211dbbb7
--- /dev/null
+++ b/drivers/raw/zxdh/zxdh_rawdev.c
@@ -0,0 +1,221 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2024 ZTE Corporation
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "zxdh_rawdev.h"
+
+/* Register offset */
+#define ZXDH_GDMA_BASE_OFFSET   0x10
+
+#define ZXDH_GDMA_CHAN_SHIFT0x80
+
+char zxdh_gdma_driver_name[] = "rawdev_zxdh_gdma";
+char dev_name[] = "zxdh_gdma";
+
+uint
+zxdh_gdma_read_reg(struct rte_rawdev *dev, uint16_t queue_id, uint offset)
+{
+   struct zxdh_gdma_rawdev *gdmadev = zxdh_gdma_rawdev_get_priv(dev);
+   uint addr = 0;
+   uint val = 0;
+
+   addr = offset + queue_id * ZXDH_GDMA_CHAN_SHIFT;
+   val = *(uint *)(gdmadev->base_addr + addr);
+
+   return val;
+}
+
+void
+zxdh_gdma_write_reg(struct rte_rawdev *dev, uint16_t queue_id, uint offset, 
uint val)
+{
+   struct zxdh_gdma_rawdev *gdmadev = zxdh_gdma_rawdev_get_priv(dev);
+   uint addr = 0;
+
+   addr = offset + queue_id * ZXDH_GDMA_CHAN_SHIFT;
+   *(uint *)(gdmadev->base_addr + addr) = val;
+}
+
+static const struct rte_rawdev_ops zxdh_gdma_rawdev_ops = {
+};
+
+static int
+zxdh_gdma_map_resource(struct rte_pci_device *dev)
+{
+   int fd = -1;
+   char devname[PATH_MAX];
+   void *mapaddr = NULL;
+   struct rte_pci_addr *loc;
+
+   loc = &dev->addr;
+   snprintf(devname, sizeof(devname), "%s/" PCI_PRI_FMT "/resource0",
+   rte_pci_get_sysfs_path(),
+   loc->domain, loc->bus, loc->devid,
+   loc->function);
+

[v1,2/5] raw/zxdh: add support for queue setup operation

[Yong Zhang]

Add queue initialization and release interface.

Signed-off-by: Yong Zhang 
---
 drivers/raw/zxdh/zxdh_rawdev.c | 241 +
 drivers/raw/zxdh/zxdh_rawdev.h |  19 +++
 2 files changed, 260 insertions(+)

diff --git a/drivers/raw/zxdh/zxdh_rawdev.c b/drivers/raw/zxdh/zxdh_rawdev.c
index 43211dbbb7..5e42669bc7 100644
--- a/drivers/raw/zxdh/zxdh_rawdev.c
+++ b/drivers/raw/zxdh/zxdh_rawdev.c
@@ -36,14 +36,58 @@
 
 #include "zxdh_rawdev.h"
 
+/*
+ * User define:
+ * ep_id-bit[15:12] vfunc_num-bit[11:4] func_num-bit[3:1] vfunc_active-bit0
+ * host ep_id:5~8   zf ep_id:9
+ */
+#define ZXDH_GDMA_ZF_USER   0x9000  /* ep4 pf0 */
+#define ZXDH_GDMA_PF_NUM_SHIFT  1
+#define ZXDH_GDMA_VF_NUM_SHIFT  4
+#define ZXDH_GDMA_EP_ID_SHIFT   12
+#define ZXDH_GDMA_VF_EN 1
+#define ZXDH_GDMA_EPID_OFFSET   5
+
 /* Register offset */
 #define ZXDH_GDMA_BASE_OFFSET   0x10
+#define ZXDH_GDMA_EXT_ADDR_OFFSET   0x218
+#define ZXDH_GDMA_CONTROL_OFFSET0x230
+#define ZXDH_GDMA_TC_CNT_OFFSET 0x23c
+#define ZXDH_GDMA_LLI_USER_OFFSET   0x228
+
+#define ZXDH_GDMA_CHAN_FORCE_CLOSE  (1 << 31)
+
+/* TC count & Error interrupt status register */
+#define ZXDH_GDMA_SRC_LLI_ERR   (1 << 16)
+#define ZXDH_GDMA_SRC_DATA_ERR  (1 << 17)
+#define ZXDH_GDMA_DST_ADDR_ERR  (1 << 18)
+#define ZXDH_GDMA_ERR_STATUS(1 << 19)
+#define ZXDH_GDMA_ERR_INTR_ENABLE   (1 << 20)
+#define ZXDH_GDMA_TC_CNT_CLEAN  (1)
 
 #define ZXDH_GDMA_CHAN_SHIFT0x80
+#define LOW32_MASK  0x
+#define LOW16_MASK  0x
+
+static int zxdh_gdma_queue_init(struct rte_rawdev *dev, uint16_t queue_id);
+static int zxdh_gdma_queue_free(struct rte_rawdev *dev, uint16_t queue_id);
 
 char zxdh_gdma_driver_name[] = "rawdev_zxdh_gdma";
 char dev_name[] = "zxdh_gdma";
 
+static inline struct zxdh_gdma_queue *
+zxdh_gdma_get_queue(struct rte_rawdev *dev, uint16_t queue_id)
+{
+   struct zxdh_gdma_rawdev *gdmadev = zxdh_gdma_rawdev_get_priv(dev);
+
+   if (queue_id >= ZXDH_GDMA_TOTAL_CHAN_NUM) {
+   ZXDH_PMD_LOG(ERR, "queue id %d is invalid", queue_id);
+   return NULL;
+   }
+
+   return &(gdmadev->vqs[queue_id]);
+}
+
 uint
 zxdh_gdma_read_reg(struct rte_rawdev *dev, uint16_t queue_id, uint offset)
 {
@@ -67,9 +111,206 @@ zxdh_gdma_write_reg(struct rte_rawdev *dev, uint16_t 
queue_id, uint offset, uint
*(uint *)(gdmadev->base_addr + addr) = val;
 }
 
+static int
+zxdh_gdma_rawdev_queue_setup(struct rte_rawdev *dev,
+uint16_t queue_id,
+rte_rawdev_obj_t 
queue_conf,
+size_t conf_size)
+{
+   struct zxdh_gdma_rawdev *gdmadev = NULL;
+   struct zxdh_gdma_queue *queue = NULL;
+   struct zxdh_gdma_queue_config *qconfig = NULL;
+   struct zxdh_gdma_rbp *rbp = NULL;
+   uint16_t i = 0;
+   uint8_t is_txq = 0;
+   uint src_user = 0;
+   uint dst_user = 0;
+
+   if (dev == NULL)
+   return -EINVAL;
+
+   if ((queue_conf == NULL) || (conf_size != sizeof(struct 
zxdh_gdma_queue_config)))
+   return -EINVAL;
+
+   gdmadev = zxdh_gdma_rawdev_get_priv(dev);
+   qconfig = (struct zxdh_gdma_queue_config *)queue_conf;
+
+   for (i = 0; i < ZXDH_GDMA_TOTAL_CHAN_NUM; i++) {
+   if (gdmadev->vqs[i].enable == 0)
+   break;
+   }
+   if (i >= ZXDH_GDMA_TOTAL_CHAN_NUM) {
+   ZXDH_PMD_LOG(ERR, "Failed to setup queue, no avail queues");
+   return -1;
+   }
+   queue_id = i;
+   if (zxdh_gdma_queue_init(dev, queue_id) != 0) {
+   ZXDH_PMD_LOG(ERR, "Failed to init queue");
+   return -1;
+   }
+   queue = &(gdmadev->vqs[queue_id]);
+
+   rbp = qconfig->rbp;
+   if ((rbp->srbp != 0) && (rbp->drbp == 0)) {
+   is_txq = 0;
+   dst_user = ZXDH_GDMA_ZF_USER;
+   src_user = ((rbp->spfid << ZXDH_GDMA_PF_NUM_SHIFT) |
+   ((rbp->sportid + ZXDH_GDMA_EPID_OFFSET) << 
ZXDH_GDMA_EP_ID_SHIFT));
+
+   if (rbp->svfid != 0)
+   src_user |= (ZXDH_GDMA_VF_EN |
+((rbp->svfid - 1) << 
ZXDH_GDMA_VF_NUM_SHIFT));
+
+   ZXDH_PMD_LOG(DEBUG, "rxq->qidx:%d setup src_user(ep:%d pf:%d 
vf:%d) success",
+   queue_id, (uint8_t)rbp->sportid, 
(uint8_t)rbp->spfid,
+   (uint8_t)rbp->svfid);
+   } else if ((rbp->srbp == 0) && (rbp->drbp != 

[v1,3/5] raw/zxdh: add support for standard rawdev operations

[Yong Zhang]

Add support for rawdev operations such as dev_start and dev_stop.

Signed-off-by: Yong Zhang 
---
 drivers/raw/zxdh/zxdh_rawdev.c | 136 -
 drivers/raw/zxdh/zxdh_rawdev.h |  10 +++
 2 files changed, 145 insertions(+), 1 deletion(-)

diff --git a/drivers/raw/zxdh/zxdh_rawdev.c b/drivers/raw/zxdh/zxdh_rawdev.c
index 5e42669bc7..ce83c3b626 100644
--- a/drivers/raw/zxdh/zxdh_rawdev.c
+++ b/drivers/raw/zxdh/zxdh_rawdev.c
@@ -111,6 +111,96 @@ zxdh_gdma_write_reg(struct rte_rawdev *dev, uint16_t 
queue_id, uint offset, uint
*(uint *)(gdmadev->base_addr + addr) = val;
 }
 
+static int
+zxdh_gdma_rawdev_info_get(struct rte_rawdev *dev,
+ __rte_unused rte_rawdev_obj_t 
dev_info,
+ __rte_unused size_t 
dev_info_size)
+{
+   if (dev == NULL)
+   return -EINVAL;
+
+   return 0;
+}
+
+static int
+zxdh_gdma_rawdev_configure(const struct rte_rawdev *dev,
+  rte_rawdev_obj_t config,
+  size_t config_size)
+{
+   struct zxdh_gdma_config *gdma_config = NULL;
+
+   if ((dev == NULL) ||
+   (config == NULL) ||
+   (config_size != sizeof(struct zxdh_gdma_config)))
+   return -EINVAL;
+
+   gdma_config = (struct zxdh_gdma_config *)config;
+   if (gdma_config->max_vqs > ZXDH_GDMA_TOTAL_CHAN_NUM) {
+   ZXDH_PMD_LOG(ERR, "gdma supports up to %d queues", 
ZXDH_GDMA_TOTAL_CHAN_NUM);
+   return -EINVAL;
+   }
+
+   return 0;
+}
+
+static int
+zxdh_gdma_rawdev_start(struct rte_rawdev *dev)
+{
+   struct zxdh_gdma_rawdev *gdmadev = NULL;
+
+   if (dev == NULL)
+   return -EINVAL;
+
+   gdmadev = zxdh_gdma_rawdev_get_priv(dev);
+   gdmadev->device_state = ZXDH_GDMA_DEV_RUNNING;
+
+   return 0;
+}
+
+static void
+zxdh_gdma_rawdev_stop(struct rte_rawdev *dev)
+{
+   struct zxdh_gdma_rawdev *gdmadev = NULL;
+
+   if (dev == NULL)
+   return;
+
+   gdmadev = zxdh_gdma_rawdev_get_priv(dev);
+   gdmadev->device_state = ZXDH_GDMA_DEV_STOPPED;
+}
+
+static int
+zxdh_gdma_rawdev_reset(struct rte_rawdev *dev)
+{
+   if (dev == NULL)
+   return -EINVAL;
+
+   return 0;
+}
+
+static int
+zxdh_gdma_rawdev_close(struct rte_rawdev *dev)
+{
+   struct zxdh_gdma_rawdev *gdmadev = NULL;
+   struct zxdh_gdma_queue *queue = NULL;
+   uint16_t queue_id = 0;
+
+   if (dev == NULL)
+   return -EINVAL;
+
+   for (queue_id = 0; queue_id < ZXDH_GDMA_TOTAL_CHAN_NUM; queue_id++) {
+   queue = zxdh_gdma_get_queue(dev, queue_id);
+   if ((queue == NULL) || (queue->enable == 0))
+   continue;
+
+   zxdh_gdma_queue_free(dev, queue_id);
+   }
+   gdmadev = zxdh_gdma_rawdev_get_priv(dev);
+   gdmadev->device_state = ZXDH_GDMA_DEV_STOPPED;
+
+   return 0;
+}
+
 static int
 zxdh_gdma_rawdev_queue_setup(struct rte_rawdev *dev,
 uint16_t queue_id,
@@ -192,8 +282,52 @@ zxdh_gdma_rawdev_queue_setup(struct rte_rawdev *dev,
return queue_id;
 }
 
+static int
+zxdh_gdma_rawdev_queue_release(struct rte_rawdev *dev, uint16_t queue_id)
+{
+   struct zxdh_gdma_queue *queue = NULL;
+
+   if (dev == NULL)
+   return -EINVAL;
+
+   queue = zxdh_gdma_get_queue(dev, queue_id);
+   if ((queue == NULL) || (queue->enable == 0))
+   return -EINVAL;
+
+   zxdh_gdma_queue_free(dev, queue_id);
+
+   return 0;
+}
+
+static int
+zxdh_gdma_rawdev_get_attr(struct rte_rawdev *dev,
+ __rte_unused const char 
*attr_name,
+ uint64_t *attr_value)
+{
+   struct zxdh_gdma_rawdev *gdmadev = NULL;
+   struct zxdh_gdma_attr *gdma_attr = NULL;
+
+   if ((dev == NULL) || (attr_value == NULL))
+   return -EINVAL;
+
+   gdmadev   = zxdh_gdma_rawdev_get_priv(dev);
+   gdma_attr = (struct zxdh_gdma_attr *)attr_value;
+   gdma_attr->num_hw_queues = gdmadev->used_num;
+
+   return 0;
+}
 static const struct rte_rawdev_ops zxdh_gdma_rawdev_ops = {
+   .dev_info_get = zxdh_gdma_rawdev_info_get,
+   .dev_configure = zxdh_gdma_rawdev_configure,
+   .dev_start = zxdh_gdma_rawdev_start,
+   .dev_stop = zxdh_gdma_rawdev_stop,
+   .dev_close = zxdh_gdma_rawdev_close,
+   .dev_reset = zxdh_gdma_rawdev_reset,
+
.queue_setup = zxdh_gdma_rawdev_queue_setup,
+   .queue_release = zxdh_gdma_rawdev_queue_release,
+
+   .attr_get = zxdh_gdma_rawdev_get_attr,
 };
 
 static int
@@ -256,7 +390,7 @@ zxdh_gdma_queue_init(struct rte_rawdev *dev, uint16_t 
queue_id)
ZXDH_PMD_LOG(INFO, "queue%u ring phy addr:0x%"PRIx64" virt addr:%p",

[v1,4/5] raw/zxdh: add support for enqueue operation

[Yong Zhang]

Add rawdev enqueue operation for zxdh devices.

Signed-off-by: Yong Zhang 
---
 drivers/raw/zxdh/zxdh_rawdev.c | 220 +
 drivers/raw/zxdh/zxdh_rawdev.h |  19 +++
 2 files changed, 239 insertions(+)

diff --git a/drivers/raw/zxdh/zxdh_rawdev.c b/drivers/raw/zxdh/zxdh_rawdev.c
index ce83c3b626..76b8f7f728 100644
--- a/drivers/raw/zxdh/zxdh_rawdev.c
+++ b/drivers/raw/zxdh/zxdh_rawdev.c
@@ -51,10 +51,34 @@
 /* Register offset */
 #define ZXDH_GDMA_BASE_OFFSET   0x10
 #define ZXDH_GDMA_EXT_ADDR_OFFSET   0x218
+#define ZXDH_GDMA_SAR_LOW_OFFSET0x200
+#define ZXDH_GDMA_DAR_LOW_OFFSET0x204
+#define ZXDH_GDMA_SAR_HIGH_OFFSET   0x234
+#define ZXDH_GDMA_DAR_HIGH_OFFSET   0x238
+#define ZXDH_GDMA_XFERSIZE_OFFSET   0x208
 #define ZXDH_GDMA_CONTROL_OFFSET0x230
+#define ZXDH_GDMA_TC_STATUS_OFFSET  0x0
+#define ZXDH_GDMA_STATUS_CLEAN_OFFSET   0x80
+#define ZXDH_GDMA_LLI_L_OFFSET  0x21c
+#define ZXDH_GDMA_LLI_H_OFFSET  0x220
+#define ZXDH_GDMA_CHAN_CONTINUE_OFFSET  0x224
 #define ZXDH_GDMA_TC_CNT_OFFSET 0x23c
 #define ZXDH_GDMA_LLI_USER_OFFSET   0x228
 
+/* Control register */
+#define ZXDH_GDMA_CHAN_ENABLE   0x1
+#define ZXDH_GDMA_CHAN_DISABLE  0
+#define ZXDH_GDMA_SOFT_CHAN 0x2
+#define ZXDH_GDMA_TC_INTR_ENABLE0x10
+#define ZXDH_GDMA_ALL_INTR_ENABLE   0x30
+#define ZXDH_GDMA_SBS_SHIFT 6   /* src burst size 
*/
+#define ZXDH_GDMA_SBL_SHIFT 9   /* src burst 
length */
+#define ZXDH_GDMA_DBS_SHIFT 13  /* dest burst size 
*/
+#define ZXDH_GDMA_BURST_SIZE_MIN0x1 /* 1 byte */
+#define ZXDH_GDMA_BURST_SIZE_MEDIUM 0x4 /* 4 word */
+#define ZXDH_GDMA_BURST_SIZE_MAX0x6 /* 16 word */
+#define ZXDH_GDMA_DEFAULT_BURST_LEN 0xf /* 16 beats */
+#define ZXDH_GDMA_TC_CNT_ENABLE (1 << 27)
 #define ZXDH_GDMA_CHAN_FORCE_CLOSE  (1 << 31)
 
 /* TC count & Error interrupt status register */
@@ -66,9 +90,15 @@
 #define ZXDH_GDMA_TC_CNT_CLEAN  (1)
 
 #define ZXDH_GDMA_CHAN_SHIFT0x80
+#define ZXDH_GDMA_LINK_END_NODE (1 << 30)
+#define ZXDH_GDMA_CHAN_CONTINUE (1)
+
 #define LOW32_MASK  0x
 #define LOW16_MASK  0x
 
+#define IDX_TO_ADDR(addr, idx, t) \
+   ((t)((uintptr_t)(addr) + (idx) * sizeof(struct zxdh_gdma_buff_desc)))
+
 static int zxdh_gdma_queue_init(struct rte_rawdev *dev, uint16_t queue_id);
 static int zxdh_gdma_queue_free(struct rte_rawdev *dev, uint16_t queue_id);
 
@@ -316,6 +346,194 @@ zxdh_gdma_rawdev_get_attr(struct rte_rawdev *dev,
 
return 0;
 }
+
+static inline void
+zxdh_gdma_control_cal(uint *val, uint8_t tc_enable)
+{
+   *val = (ZXDH_GDMA_CHAN_ENABLE |
+   ZXDH_GDMA_SOFT_CHAN |
+   (ZXDH_GDMA_DEFAULT_BURST_LEN << ZXDH_GDMA_SBL_SHIFT) |
+   (ZXDH_GDMA_BURST_SIZE_MAX << ZXDH_GDMA_SBS_SHIFT) |
+   (ZXDH_GDMA_BURST_SIZE_MAX << ZXDH_GDMA_DBS_SHIFT));
+
+   if (tc_enable != 0)
+   *val |= ZXDH_GDMA_TC_CNT_ENABLE;
+}
+
+static inline uint
+zxdh_gdma_user_get(struct zxdh_gdma_queue *queue, struct zxdh_gdma_job *job)
+{
+   uint src_user = 0;
+   uint dst_user = 0;
+
+   if ((job->flags & ZXDH_GDMA_JOB_DIR_MASK) == 0) {
+   ZXDH_PMD_LOG(DEBUG, "job flags:0x%x default user:0x%x",
+   job->flags, 
queue->user);
+   return queue->user;
+   } else if ((job->flags & ZXDH_GDMA_JOB_DIR_TX) != 0) {
+   src_user = ZXDH_GDMA_ZF_USER;
+   dst_user = ((job->pf_id << ZXDH_GDMA_PF_NUM_SHIFT) |
+   ((job->ep_id + ZXDH_GDMA_EPID_OFFSET) << 
ZXDH_GDMA_EP_ID_SHIFT));
+
+   if (job->vf_id != 0)
+   dst_user |= (ZXDH_GDMA_VF_EN |
+((job->vf_id - 1) << 
ZXDH_GDMA_VF_NUM_SHIFT));
+   } else {
+   dst_user = ZXDH_GDMA_ZF_USER;
+   src_user = ((job->pf_id << ZXDH_GDMA_PF_NUM_SHIFT) |
+   ((job->ep_id + ZXDH_GDMA_EPID_OFFSET) << 
ZXDH_GDMA_EP_ID_SHIFT));
+
+   if (job->vf_id != 0)
+   src_user |= (ZXDH_GDMA_VF_EN |
+((job->vf_id - 1) << 
ZXDH_GDMA_VF_NUM_SHIFT));
+   }
+   ZXDH_PMD_LOG(DEBUG, "job flags:0x%x ep_id:%u, pf_id:%u, vf_id:%u, 
user:0x%x",
+   job->flags, job->ep_id, 
job->pf_id, job->vf_id,
+

[v1,5/5] raw/zxdh: add support for dequeue operation

[Yong Zhang]

Add rawdev dequeue operation for zxdh devices.

Signed-off-by: Yong Zhang 
---
 drivers/raw/zxdh/zxdh_rawdev.c | 113 +
 1 file changed, 113 insertions(+)

diff --git a/drivers/raw/zxdh/zxdh_rawdev.c b/drivers/raw/zxdh/zxdh_rawdev.c
index 76b8f7f728..2019bd3704 100644
--- a/drivers/raw/zxdh/zxdh_rawdev.c
+++ b/drivers/raw/zxdh/zxdh_rawdev.c
@@ -96,6 +96,8 @@
 #define LOW32_MASK  0x
 #define LOW16_MASK  0x
 
+#define ZXDH_GDMA_TC_CNT_MAX0x1
+
 #define IDX_TO_ADDR(addr, idx, t) \
((t)((uintptr_t)(addr) + (idx) * sizeof(struct zxdh_gdma_buff_desc)))
 
@@ -534,6 +536,116 @@ zxdh_gdma_rawdev_enqueue_bufs(struct rte_rawdev *dev,
 
return count;
 }
+
+static inline void
+zxdh_gdma_used_idx_update(struct zxdh_gdma_queue *queue, uint16_t cnt, uint8_t 
data_bd_err)
+{
+   uint16_t idx = 0;
+
+   if (queue->sw_ring.used_idx + cnt < queue->queue_size)
+   queue->sw_ring.used_idx += cnt;
+   else
+   queue->sw_ring.used_idx = queue->sw_ring.used_idx + cnt - 
queue->queue_size;
+
+   if (data_bd_err == 1) {
+   /* Update job status, the last job status is error */
+   if (queue->sw_ring.used_idx == 0)
+   idx = queue->queue_size - 1;
+   else
+   idx = queue->sw_ring.used_idx - 1;
+
+   queue->sw_ring.job[idx]->status = 1;
+   }
+}
+
+static int
+zxdh_gdma_rawdev_dequeue_bufs(struct rte_rawdev *dev,
+   __rte_unused struct 
rte_rawdev_buf **buffers,
+   uint count,
+   rte_rawdev_obj_t context)
+{
+   struct zxdh_gdma_queue *queue = NULL;
+   struct zxdh_gdma_enqdeq *e_context = NULL;
+   uint16_t queue_id = 0;
+   uint val = 0;
+   uint16_t tc_cnt = 0;
+   uint16_t diff_cnt = 0;
+   uint16_t i = 0;
+   uint16_t bd_idx = 0;
+   uint64_t next_bd_addr = 0;
+   uint8_t data_bd_err = 0;
+
+   if ((dev == NULL) || (context == NULL))
+   return -EINVAL;
+
+   e_context = (struct zxdh_gdma_enqdeq *)context;
+   queue_id = e_context->vq_id;
+   queue = zxdh_gdma_get_queue(dev, queue_id);
+   if ((queue == NULL) || (queue->enable == 0))
+   return -EINVAL;
+
+   if (queue->sw_ring.pend_cnt == 0)
+   goto deq_job;
+
+   /* Get data transmit count */
+   val = zxdh_gdma_read_reg(dev, queue_id, ZXDH_GDMA_TC_CNT_OFFSET);
+   tc_cnt = val & LOW16_MASK;
+   if (tc_cnt >= queue->tc_cnt)
+   diff_cnt = tc_cnt - queue->tc_cnt;
+   else
+   diff_cnt = tc_cnt + ZXDH_GDMA_TC_CNT_MAX - queue->tc_cnt;
+
+   queue->tc_cnt = tc_cnt;
+
+   /* Data transmit error, channel stopped */
+   if ((val & ZXDH_GDMA_ERR_STATUS) != 0) {
+   next_bd_addr  = zxdh_gdma_read_reg(dev, queue_id, 
ZXDH_GDMA_LLI_L_OFFSET);
+   next_bd_addr |= ((uint64_t)zxdh_gdma_read_reg(dev, queue_id,
+   ZXDH_GDMA_LLI_H_OFFSET) 
<< 32);
+   next_bd_addr  = next_bd_addr << 6;
+   bd_idx = (next_bd_addr - queue->ring.ring_mem) / sizeof(struct 
zxdh_gdma_buff_desc);
+   if ((val & ZXDH_GDMA_SRC_DATA_ERR) || (val & 
ZXDH_GDMA_DST_ADDR_ERR)) {
+   diff_cnt++;
+   data_bd_err = 1;
+   }
+   ZXDH_PMD_LOG(INFO, "queue%d is err(0x%x) next_bd_idx:%u 
ll_addr:0x%"PRIx64" def user:0x%x",
+   queue_id, val, bd_idx, next_bd_addr, 
queue->user);
+
+   ZXDH_PMD_LOG(INFO, "Clean up error status");
+   val = ZXDH_GDMA_ERR_STATUS | ZXDH_GDMA_ERR_INTR_ENABLE;
+   zxdh_gdma_write_reg(dev, queue_id, ZXDH_GDMA_TC_CNT_OFFSET, 
val);
+
+   ZXDH_PMD_LOG(INFO, "Restart channel");
+   zxdh_gdma_write_reg(dev, queue_id, ZXDH_GDMA_XFERSIZE_OFFSET, 
0);
+   zxdh_gdma_control_cal(&val, 0);
+   zxdh_gdma_write_reg(dev, queue_id, ZXDH_GDMA_CONTROL_OFFSET, 
val);
+   }
+
+   if (diff_cnt != 0) {
+   zxdh_gdma_used_idx_update(queue, diff_cnt, data_bd_err);
+   queue->sw_ring.deq_cnt += diff_cnt;
+   queue->sw_ring.pend_cnt -= diff_cnt;
+   }
+
+deq_job:
+   if (queue->sw_ring.deq_cnt == 0)
+   return 0;
+   else if (queue->sw_ring.deq_cnt < count)
+   count = queue->sw_ring.deq_cnt;
+
+   queue->sw_ring.deq_cnt -= count;
+
+   for (i = 0; i < count; i++) {
+   e_context->job[i] = queue->sw_ring.job[queue->sw_ring.deq_idx];
+   queue->sw_ring.job[queue->sw_ring.deq_idx] = NULL;
+   if (++queue->sw_ring.deq_idx >= queue->queue_size)
+ 

[PATCH] app/dma-perf: per device config support

[Amit Prakash Shukla]

Add support to configure device specific config parameters for a
testcase. Example:

lcore_dma0=lcore=11,dev=:00:04.1,dir=mem2dev,raddr=0x3,
coreid=1,pfid=2,vfid=3
lcore_dma1=lcore=12,dev=:00:04.2,dir=dev2mem,raddr=0x2,
coreid=3,pfid=2,vfid=1

Signed-off-by: Amit Prakash Shukla 
---
 app/test-dma-perf/benchmark.c | 318 +-
 app/test-dma-perf/config.ini  |  30 ++--
 app/test-dma-perf/main.c  | 199 ++---
 app/test-dma-perf/main.h  |  20 ++-
 doc/guides/tools/dmaperf.rst  | 107 +---
 5 files changed, 360 insertions(+), 314 deletions(-)

diff --git a/app/test-dma-perf/benchmark.c b/app/test-dma-perf/benchmark.c
index d167adc4d2..5eebb4de77 100644
--- a/app/test-dma-perf/benchmark.c
+++ b/app/test-dma-perf/benchmark.c
@@ -148,11 +148,13 @@ cache_flush_buf(__rte_unused struct rte_mbuf **array,
 
 static int
 vchan_data_populate(uint32_t dev_id, struct rte_dma_vchan_conf *qconf,
-   struct test_configure *cfg)
+   struct test_configure *cfg, uint16_t dev_num)
 {
+   struct vchan_dev_config *vchan_dconfig;
struct rte_dma_info info;
 
-   qconf->direction = cfg->transfer_dir;
+   vchan_dconfig = &cfg->dma_config[dev_num].vchan_dev;
+   qconf->direction = vchan_dconfig->tdir;
 
rte_dma_info_get(dev_id, &info);
if (!(RTE_BIT64(qconf->direction) & info.dev_capa))
@@ -164,16 +166,16 @@ vchan_data_populate(uint32_t dev_id, struct 
rte_dma_vchan_conf *qconf,
case RTE_DMA_DIR_MEM_TO_DEV:
qconf->dst_port.pcie.vfen = 1;
qconf->dst_port.port_type = RTE_DMA_PORT_PCIE;
-   qconf->dst_port.pcie.coreid = cfg->vchan_dev.port.pcie.coreid;
-   qconf->dst_port.pcie.vfid = cfg->vchan_dev.port.pcie.vfid;
-   qconf->dst_port.pcie.pfid = cfg->vchan_dev.port.pcie.pfid;
+   qconf->dst_port.pcie.coreid = vchan_dconfig->port.pcie.coreid;
+   qconf->dst_port.pcie.vfid = vchan_dconfig->port.pcie.vfid;
+   qconf->dst_port.pcie.pfid = vchan_dconfig->port.pcie.pfid;
break;
case RTE_DMA_DIR_DEV_TO_MEM:
qconf->src_port.pcie.vfen = 1;
qconf->src_port.port_type = RTE_DMA_PORT_PCIE;
-   qconf->src_port.pcie.coreid = cfg->vchan_dev.port.pcie.coreid;
-   qconf->src_port.pcie.vfid = cfg->vchan_dev.port.pcie.vfid;
-   qconf->src_port.pcie.pfid = cfg->vchan_dev.port.pcie.pfid;
+   qconf->src_port.pcie.coreid = vchan_dconfig->port.pcie.coreid;
+   qconf->src_port.pcie.vfid = vchan_dconfig->port.pcie.vfid;
+   qconf->src_port.pcie.pfid = vchan_dconfig->port.pcie.pfid;
break;
case RTE_DMA_DIR_MEM_TO_MEM:
case RTE_DMA_DIR_DEV_TO_DEV:
@@ -185,14 +187,15 @@ vchan_data_populate(uint32_t dev_id, struct 
rte_dma_vchan_conf *qconf,
 
 /* Configuration of device. */
 static void
-configure_dmadev_queue(uint32_t dev_id, struct test_configure *cfg, uint8_t 
sges_max)
+configure_dmadev_queue(uint32_t dev_id, struct test_configure *cfg, uint8_t 
sges_max,
+  uint16_t dev_num)
 {
uint16_t vchan = 0;
struct rte_dma_info info;
struct rte_dma_conf dev_config = { .nb_vchans = 1 };
struct rte_dma_vchan_conf qconf = { 0 };
 
-   if (vchan_data_populate(dev_id, &qconf, cfg) != 0)
+   if (vchan_data_populate(dev_id, &qconf, cfg, dev_num) != 0)
rte_exit(EXIT_FAILURE, "Error with vchan data populate.\n");
 
if (rte_dma_configure(dev_id, &dev_config) != 0)
@@ -219,8 +222,8 @@ configure_dmadev_queue(uint32_t dev_id, struct 
test_configure *cfg, uint8_t sges
 static int
 config_dmadevs(struct test_configure *cfg)
 {
-   struct lcore_dma_map_t *ldm = &cfg->lcore_dma_map;
-   uint32_t nb_workers = ldm->cnt;
+   uint32_t nb_workers = cfg->num_worker;
+   struct lcore_dma_map_t *ldm;
uint32_t i;
int dev_id;
uint16_t nb_dmadevs = 0;
@@ -230,16 +233,17 @@ config_dmadevs(struct test_configure *cfg)
if (cfg->is_sg)
nb_sges = RTE_MAX(cfg->nb_src_sges, cfg->nb_dst_sges);
 
-   for (i = 0; i < ldm->cnt; i++) {
-   dma_name = ldm->dma_names[i];
+   for (i = 0; i < nb_workers; i++) {
+   ldm = &cfg->dma_config[i].lcore_dma_map;
+   dma_name = ldm->dma_names;
dev_id = rte_dma_get_dev_id_by_name(dma_name);
if (dev_id < 0) {
fprintf(stderr, "Error: Fail to find DMA %s.\n", 
dma_name);
goto end;
}
 
-   ldm->dma_ids[i] = dev_id;
-   configure_dmadev_queue(dev_id, cfg, nb_sges);
+   ldm->dma_id = dev_id;
+   configure_dmadev_queue(dev_id, cfg, nb_sges, nb_dmadevs);
++nb_dmadevs;
}
 
@@ -436,7 +440,6 @@ setup_memory_env(struct

[PATCH v11 0/5] dts: API docs generation

[Juraj Linkeš]

The generation is done with Sphinx, which DPDK already uses, with
slightly modified configuration of the sidebar present in an if block.

DTS dependencies do not need to be installed, but there is the option to
install doc build dependencies with Poetry:
poetry install --with docs

The build itself may be run with:
meson setup  -Denable_docs=true
ninja -C 

The above will do a full DPDK build with docs. To build just docs:
meson setup 
ninja -C  dts-doc

Python3.10 is required to build the DTS API docs.

The patchset contains the .rst sources which Sphinx uses to generate the
html pages. These were first generated with the sphinx-apidoc utility
and modified to provide a better look. The documentation just doesn't
look that good without the modifications and there isn't enough
configuration options to achieve that without manual changes to the .rst
files. This introduces extra maintenance which involves adding new .rst
files when a new Python module is added or changing the .rst structure
if the Python directory/file structure is changed (moved, renamed
files). This small maintenance burden is outweighed by the flexibility
afforded by the ability to make manual changes to the .rst files.

v2:
Removed the use of sphinx-apidoc from meson in favor of adding the files
generated by it directly to the repository (and modifying them).

v3:
Rebase.

v4:
Rebase.

v5:
Another rebase, but this time the rebase needed the addition of .rst
corresponding to newly added files as well as fixing a few documentation
problems in said files.

v6:
Documentation formatting adjustments.

v7:
Now with the actual doc changes.

v8:
Split the last commit into non-DTS and DTS changes.

v9:
Rebase.

v10:
Fix dts doc generation issue: Only copy the custom rss file if it exists.

v11:
Added the config option autodoc_mock_imports, which eliminates the need
for DTS dependencies. Added a script that find out which imports need to
be added to autodoc_mock_imports. The script also check the required
Python version for building DTS docs.
Removed tags from the two affected patches which will need to be
reviewed again.

Juraj Linkeš (5):
  dts: update params and parser docstrings
  dts: add doc generation dependencies
  dts: add API doc sources
  doc: meson doc API build dir variable
  dts: add API doc generation

 buildtools/call-sphinx-build.py   |  10 +-
 buildtools/get-dts-deps.py|  78 +++
 buildtools/meson.build|   1 +
 doc/api/doxy-api-index.md |   3 +
 doc/api/doxy-api.conf.in  |   2 +
 doc/api/meson.build   |   8 +-
 doc/guides/conf.py|  41 +-
 doc/guides/contributing/documentation.rst |   2 +
 doc/guides/contributing/patches.rst   |   4 +
 doc/guides/meson.build|   1 +
 doc/guides/tools/dts.rst  |  39 +-
 dts/doc/conf_yaml_schema.json |   1 +
 dts/doc/framework.config.rst  |  12 +
 dts/doc/framework.config.types.rst|   6 +
 dts/doc/framework.exception.rst   |   6 +
 dts/doc/framework.logger.rst  |   6 +
 dts/doc/framework.params.eal.rst  |   6 +
 dts/doc/framework.params.rst  |  14 +
 dts/doc/framework.params.testpmd.rst  |   6 +
 dts/doc/framework.params.types.rst|   6 +
 dts/doc/framework.parser.rst  |   6 +
 .../framework.remote_session.dpdk_shell.rst   |   6 +
 ...ote_session.interactive_remote_session.rst |   6 +
 ...ework.remote_session.interactive_shell.rst |   6 +
 .../framework.remote_session.python_shell.rst |   6 +
 ...ramework.remote_session.remote_session.rst |   6 +
 dts/doc/framework.remote_session.rst  |  18 +
 .../framework.remote_session.ssh_session.rst  |   6 +
 ...framework.remote_session.testpmd_shell.rst |   6 +
 dts/doc/framework.runner.rst  |   6 +
 dts/doc/framework.settings.rst|   6 +
 dts/doc/framework.test_result.rst |   6 +
 dts/doc/framework.test_suite.rst  |   6 +
 dts/doc/framework.testbed_model.cpu.rst   |   6 +
 .../framework.testbed_model.linux_session.rst |   6 +
 dts/doc/framework.testbed_model.node.rst  |   6 +
 .../framework.testbed_model.os_session.rst|   6 +
 dts/doc/framework.testbed_model.port.rst  |   6 +
 .../framework.testbed_model.posix_session.rst |   6 +
 dts/doc/framework.testbed_model.rst   |  26 +
 dts/doc/framework.testbed_model.sut_node.rst  |   6 +
 dts/doc/framework.testbed_model.tg_node.rst   |   6 +
 ..._generator.capturing_traffic_generator.rst |   6 +
 ...mework.testbed_model.traffic_generator.rst |  14 +
 testbed_model.traffic_generator.scapy.rst |   6 +
 ...el.traffic_generator.traffic_generator.rst |   6 +
 ...framework.testbed_model.virtual_device.rst |   6 +
 dts/doc/framework.utils.rst   |   6 +
 dts/doc/index.rst

[PATCH v11 1/5] dts: update params and parser docstrings

[Juraj Linkeš]

Address a few errors reported by Sphinx when generating documentation:
framework/params/__init__.py:docstring of framework.params.modify_str:3:
WARNING: Inline interpreted text or phrase reference start-string
without end-string.
framework/params/eal.py:docstring of framework.params.eal.EalParams:35:
WARNING: Definition list ends without a blank line; unexpected
unindent.
framework/params/types.py:docstring of framework.params.types:8:
WARNING: Inline strong start-string without end-string.
framework/params/types.py:docstring of framework.params.types:9:
WARNING: Inline strong start-string without end-string.
framework/parser.py:docstring of framework.parser.TextParser:33: ERROR:
Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser:43: ERROR:
Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser:49: ERROR:
Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser.wrap:8:
ERROR: Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser.wrap:9:
WARNING: Block quote ends without a blank line; unexpected unindent.

Fixes: 87ba4cdc0dbb ("dts: use Unpack for type checking and hinting")
Fixes: d70159cb62f5 ("dts: add params manipulation module")
Fixes: 967fc62b0a43 ("dts: refactor EAL parameters class")
Fixes: 818fe14e3422 ("dts: add parsing utility module")
Cc: luca.vizza...@arm.com

Signed-off-by: Juraj Linkeš 
---
 dts/framework/params/__init__.py | 4 ++--
 dts/framework/params/eal.py  | 7 +--
 dts/framework/params/types.py| 3 ++-
 dts/framework/parser.py  | 4 ++--
 4 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/dts/framework/params/__init__.py b/dts/framework/params/__init__.py
index 5a6fd93053..1ae227d7b4 100644
--- a/dts/framework/params/__init__.py
+++ b/dts/framework/params/__init__.py
@@ -53,9 +53,9 @@ def reduced_fn(value):
 
 
 def modify_str(*funcs: FnPtr) -> Callable[[T], T]:
-"""Class decorator modifying the ``__str__`` method with a function 
created from its arguments.
+r"""Class decorator modifying the ``__str__`` method with a function 
created from its arguments.
 
-The :attr:`FnPtr`s fed to the decorator are executed from left to right in 
the arguments list
+The :attr:`FnPtr`\s fed to the decorator are executed from left to right 
in the arguments list
 order.
 
 Args:
diff --git a/dts/framework/params/eal.py b/dts/framework/params/eal.py
index 8d7766fefc..cf1594353a 100644
--- a/dts/framework/params/eal.py
+++ b/dts/framework/params/eal.py
@@ -26,13 +26,16 @@ class EalParams(Params):
 prefix: Set the file prefix string with which to start DPDK, e.g.: 
``prefix="vf"``.
 no_pci: Switch to disable PCI bus, e.g.: ``no_pci=True``.
 vdevs: Virtual devices, e.g.::
+
 vdevs=[
 VirtualDevice('net_ring0'),
 VirtualDevice('net_ring1')
 ]
+
 ports: The list of ports to allow.
-other_eal_param: user defined DPDK EAL parameters, e.g.:
-``other_eal_param='--single-file-segments'``
+other_eal_param: user defined DPDK EAL parameters, e.g.::
+
+``other_eal_param='--single-file-segments'``
 """
 
 lcore_list: LogicalCoreList | None = field(default=None, 
metadata=Params.short("l"))
diff --git a/dts/framework/params/types.py b/dts/framework/params/types.py
index e668f658d8..d77c4625fb 100644
--- a/dts/framework/params/types.py
+++ b/dts/framework/params/types.py
@@ -6,7 +6,8 @@
 TypedDicts can be used in conjunction with Unpack and kwargs for type hinting 
on function calls.
 
 Example:
-..code:: python
+.. code:: python
+
 def create_testpmd(**kwargs: Unpack[TestPmdParamsDict]):
 params = TestPmdParams(**kwargs)
 """
diff --git a/dts/framework/parser.py b/dts/framework/parser.py
index 741dfff821..7254c75b71 100644
--- a/dts/framework/parser.py
+++ b/dts/framework/parser.py
@@ -46,7 +46,7 @@ class TextParser(ABC):
 Example:
 The following example makes use of and demonstrates every parser 
function available:
 
-..code:: python
+.. code:: python
 
 from dataclasses import dataclass, field
 from enum import Enum
@@ -90,7 +90,7 @@ def wrap(parser_fn: ParserFn, wrapper_fn: Callable) -> 
ParserFn:
 """Makes a wrapped parser function.
 
 `parser_fn` is called and if a non-None value is returned, 
`wrapper_function` is called with
-it. Otherwise the function returns early with None. In pseudo-code:
+it. Otherwise the function returns early with None. In pseudo-code::
 
 intermediate_value := parser_fn(input)
 if intermediary_value is None then
-- 
2.34.1

[PATCH v11 2/5] dts: add doc generation dependencies

[Juraj Linkeš]

Sphinx imports every Python module (through the autodoc extension)
when generating documentation from docstrings, meaning all DTS
dependencies, including Python version, should be satisfied. This is not
a hard requirement, as imports from dependencies may be mocked in the
autodoc_mock_imports autodoc option.
In case DTS developers want to use a Sphinx installation from their
virtualenv, we provide an optional Poetry group for doc generation. The
pyelftools package is there so that meson picks up the correct Python
installation, as pyelftools is required by the build system.

Signed-off-by: Juraj Linkeš 
---
 dts/poetry.lock| 521 +++--
 dts/pyproject.toml |   8 +
 2 files changed, 517 insertions(+), 12 deletions(-)

diff --git a/dts/poetry.lock b/dts/poetry.lock
index 5f8fa03933..2dd8bad498 100644
--- a/dts/poetry.lock
+++ b/dts/poetry.lock
@@ -1,5 +1,16 @@
 # This file is automatically @generated by Poetry 1.8.2 and should not be 
changed by hand.
 
+[[package]]
+name = "alabaster"
+version = "0.7.13"
+description = "A configurable sidebar-enabled Sphinx theme"
+optional = false
+python-versions = ">=3.6"
+files = [
+{file = "alabaster-0.7.13-py3-none-any.whl", hash = 
"sha256:1ee19aca801bbabb5ba3f5f258e4422dfa86f82f3e9cefb0859b283cdd7f62a3"},
+{file = "alabaster-0.7.13.tar.gz", hash = 
"sha256:a27a4a084d5e690e16e01e03ad2b2e552c61a65469419b907243193de1a84ae2"},
+]
+
 [[package]]
 name = "attrs"
 version = "23.1.0"
@@ -18,6 +29,23 @@ docs = ["furo", "myst-parser", "sphinx", 
"sphinx-notfound-page", "sphinxcontrib-
 tests = ["attrs[tests-no-zope]", "zope-interface"]
 tests-no-zope = ["cloudpickle", "hypothesis", "mypy (>=1.1.1)", "pympler", 
"pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"]
 
+[[package]]
+name = "babel"
+version = "2.13.1"
+description = "Internationalization utilities"
+optional = false
+python-versions = ">=3.7"
+files = [
+{file = "Babel-2.13.1-py3-none-any.whl", hash = 
"sha256:7077a4984b02b6727ac10f1f7294484f737443d7e2e66c5e4380e41a3ae0b4ed"},
+{file = "Babel-2.13.1.tar.gz", hash = 
"sha256:33e0952d7dd6374af8dbf6768cc4ddf3ccfefc244f9986d4074704f2fbd18900"},
+]
+
+[package.dependencies]
+setuptools = {version = "*", markers = "python_version >= \"3.12\""}
+
+[package.extras]
+dev = ["freezegun (>=1.0,<2.0)", "pytest (>=6.0)", "pytest-cov"]
+
 [[package]]
 name = "bcrypt"
 version = "4.0.1"
@@ -86,6 +114,17 @@ d = ["aiohttp (>=3.7.4)"]
 jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
 uvloop = ["uvloop (>=0.15.2)"]
 
+[[package]]
+name = "certifi"
+version = "2023.7.22"
+description = "Python package for providing Mozilla's CA Bundle."
+optional = false
+python-versions = ">=3.6"
+files = [
+{file = "certifi-2023.7.22-py3-none-any.whl", hash = 
"sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9"},
+{file = "certifi-2023.7.22.tar.gz", hash = 
"sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082"},
+]
+
 [[package]]
 name = "cffi"
 version = "1.15.1"
@@ -162,6 +201,105 @@ files = [
 [package.dependencies]
 pycparser = "*"
 
+[[package]]
+name = "charset-normalizer"
+version = "3.3.2"
+description = "The Real First Universal Charset Detector. Open, modern and 
actively maintained alternative to Chardet."
+optional = false
+python-versions = ">=3.7.0"
+files = [
+{file = "charset-normalizer-3.3.2.tar.gz", hash = 
"sha256:f30c3cb33b24454a82faecaf01b19c18562b1e89558fb6c56de4d9118a032fd5"},
+{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_universal2.whl", 
hash = 
"sha256:25baf083bf6f6b341f4121c2f3c548875ee6f5339300e08be3f2b2ba1721cdd3"},
+{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_x86_64.whl", 
hash = 
"sha256:06435b539f889b1f6f4ac1758871aae42dc3a8c0e24ac9e60c2384973ad73027"},
+{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl", hash 
= "sha256:9063e24fdb1e498ab71cb7419e24622516c4a04476b17a2dab57e8baa30d6e03"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
 hash = 
"sha256:6897af51655e3691ff853668779c7bad41579facacf5fd7253b0133308cf000d"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl",
 hash = 
"sha256:1d3193f4a680c64b4b6a9115943538edb896edc190f0b222e73761716519268e"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl",
 hash = 
"sha256:cd70574b12bb8a4d2aaa0094515df2463cb429d8536cfb6c7ce983246983e5a6"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
 hash = 
"sha256:8465322196c8b4d7ab6d1e049e4c5cb460d0394da4a27d23cc242fbf0034b6b5"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl",
 hash = 
"sha256:a9a8e9031d613fd2009c182b69c7b2c1ef8239a0efb1df3f7c8da66d5dd3d537"},
+{file = "charset_normalizer-3.3.2-cp310-cp

[PATCH v11 3/5] dts: add API doc sources

[Juraj Linkeš]

These sources could be generated with the sphinx-apidoc utility, but
that doesn't give us enough flexibility, such as sorting the order of
modules or changing the headers of the modules.

The sources included in this patch were in fact generated by said
utility, but modified to improve the look of the documentation. The
improvements are mainly in toctree definitions and the titles of the
modules/packages. These were made with specific Sphinx config options in
mind.

Signed-off-by: Juraj Linkeš 
Reviewed-by: Luca Vizzarro 
Reviewed-by: Jeremy Spewock 
Tested-by: Luca Vizzarro 
---
 dts/doc/conf_yaml_schema.json |  1 +
 dts/doc/framework.config.rst  | 12 ++
 dts/doc/framework.config.types.rst|  6 +++
 dts/doc/framework.exception.rst   |  6 +++
 dts/doc/framework.logger.rst  |  6 +++
 dts/doc/framework.params.eal.rst  |  6 +++
 dts/doc/framework.params.rst  | 14 ++
 dts/doc/framework.params.testpmd.rst  |  6 +++
 dts/doc/framework.params.types.rst|  6 +++
 dts/doc/framework.parser.rst  |  6 +++
 .../framework.remote_session.dpdk_shell.rst   |  6 +++
 ...ote_session.interactive_remote_session.rst |  6 +++
 ...ework.remote_session.interactive_shell.rst |  6 +++
 .../framework.remote_session.python_shell.rst |  6 +++
 ...ramework.remote_session.remote_session.rst |  6 +++
 dts/doc/framework.remote_session.rst  | 18 
 .../framework.remote_session.ssh_session.rst  |  6 +++
 ...framework.remote_session.testpmd_shell.rst |  6 +++
 dts/doc/framework.runner.rst  |  6 +++
 dts/doc/framework.settings.rst|  6 +++
 dts/doc/framework.test_result.rst |  6 +++
 dts/doc/framework.test_suite.rst  |  6 +++
 dts/doc/framework.testbed_model.cpu.rst   |  6 +++
 .../framework.testbed_model.linux_session.rst |  6 +++
 dts/doc/framework.testbed_model.node.rst  |  6 +++
 .../framework.testbed_model.os_session.rst|  6 +++
 dts/doc/framework.testbed_model.port.rst  |  6 +++
 .../framework.testbed_model.posix_session.rst |  6 +++
 dts/doc/framework.testbed_model.rst   | 26 +++
 dts/doc/framework.testbed_model.sut_node.rst  |  6 +++
 dts/doc/framework.testbed_model.tg_node.rst   |  6 +++
 ..._generator.capturing_traffic_generator.rst |  6 +++
 ...mework.testbed_model.traffic_generator.rst | 14 ++
 testbed_model.traffic_generator.scapy.rst |  6 +++
 ...el.traffic_generator.traffic_generator.rst |  6 +++
 ...framework.testbed_model.virtual_device.rst |  6 +++
 dts/doc/framework.utils.rst   |  6 +++
 dts/doc/index.rst | 43 +++
 38 files changed, 314 insertions(+)
 create mode 12 dts/doc/conf_yaml_schema.json
 create mode 100644 dts/doc/framework.config.rst
 create mode 100644 dts/doc/framework.config.types.rst
 create mode 100644 dts/doc/framework.exception.rst
 create mode 100644 dts/doc/framework.logger.rst
 create mode 100644 dts/doc/framework.params.eal.rst
 create mode 100644 dts/doc/framework.params.rst
 create mode 100644 dts/doc/framework.params.testpmd.rst
 create mode 100644 dts/doc/framework.params.types.rst
 create mode 100644 dts/doc/framework.parser.rst
 create mode 100644 dts/doc/framework.remote_session.dpdk_shell.rst
 create mode 100644 
dts/doc/framework.remote_session.interactive_remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.interactive_shell.rst
 create mode 100644 dts/doc/framework.remote_session.python_shell.rst
 create mode 100644 dts/doc/framework.remote_session.remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.ssh_session.rst
 create mode 100644 dts/doc/framework.remote_session.testpmd_shell.rst
 create mode 100644 dts/doc/framework.runner.rst
 create mode 100644 dts/doc/framework.settings.rst
 create mode 100644 dts/doc/framework.test_result.rst
 create mode 100644 dts/doc/framework.test_suite.rst
 create mode 100644 dts/doc/framework.testbed_model.cpu.rst
 create mode 100644 dts/doc/framework.testbed_model.linux_session.rst
 create mode 100644 dts/doc/framework.testbed_model.node.rst
 create mode 100644 dts/doc/framework.testbed_model.os_session.rst
 create mode 100644 dts/doc/framework.testbed_model.port.rst
 create mode 100644 dts/doc/framework.testbed_model.posix_session.rst
 create mode 100644 dts/doc/framework.testbed_model.rst
 create mode 100644 dts/doc/framework.testbed_model.sut_node.rst
 create mode 100644 dts/doc/framework.testbed_model.tg_node.rst
 create mode 100644 
dts/doc/framework.testbed_model.traffic_generator.capturing_traffic_generator.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.scapy.rst
 create mode 100644 
dts/doc/framework.testbed_model.traffic_generator.traffic_generator

[PATCH v11 4/5] doc: meson doc API build dir variable

[Juraj Linkeš]

The three instances of the path 'dpdk_build_root/doc/api' are replaced
with a variable, moving the definition to one place.

Signed-off-by: Juraj Linkeš 
Reviewed-by: Luca Vizzarro 
Reviewed-by: Jeremy Spewock 
Acked-by: Bruce Richardson 
Tested-by: Luca Vizzarro 
Tested-by: Nicholas Pratte 
---
 doc/api/meson.build | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 5b50692df9..b828b1ed66 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi 
 
+doc_api_build_dir = meson.current_build_dir()
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
 if not doxygen.found()
@@ -32,10 +33,10 @@ example = custom_target('examples.dox',
 # set up common Doxygen configuration
 cdata = configuration_data()
 cdata.set('VERSION', meson.project_version())
-cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 
'examples.dox'))
-cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
+cdata.set('API_EXAMPLES', join_paths(doc_api_build_dir, 'examples.dox'))
+cdata.set('OUTPUT', doc_api_build_dir)
 cdata.set('TOPDIR', dpdk_source_root)
-cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, 
join_paths(dpdk_build_root, 'doc', 'api')]))
+cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, doc_api_build_dir]))
 cdata.set('WARN_AS_ERROR', 'NO')
 if get_option('werror')
 cdata.set('WARN_AS_ERROR', 'YES')
-- 
2.34.1

[PATCH v11 5/5] dts: add API doc generation

[Juraj Linkeš]

The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.

Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.

There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.

The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.

[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings

Signed-off-by: Juraj Linkeš 
---
 buildtools/call-sphinx-build.py   | 10 ++-
 buildtools/get-dts-deps.py| 78 +++
 buildtools/meson.build|  1 +
 doc/api/doxy-api-index.md |  3 +
 doc/api/doxy-api.conf.in  |  2 +
 doc/api/meson.build   |  1 +
 doc/guides/conf.py| 41 +++-
 doc/guides/contributing/documentation.rst |  2 +
 doc/guides/contributing/patches.rst   |  4 ++
 doc/guides/meson.build|  1 +
 doc/guides/tools/dts.rst  | 39 +++-
 dts/doc/meson.build   | 30 +
 dts/meson.build   | 15 +
 meson.build   |  1 +
 14 files changed, 225 insertions(+), 3 deletions(-)
 create mode 100755 buildtools/get-dts-deps.py
 create mode 100644 dts/doc/meson.build
 create mode 100644 dts/meson.build

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..5dd59907cd 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,11 @@
 
 # set the version in environment for sphinx to pick up
 os.environ['DPDK_VERSION'] = version
+conf_src = src
+if src.find('dts') != -1:
+if '-c' in extra_args:
+conf_src = extra_args[extra_args.index('-c') + 1]
+os.environ['DTS_BUILD'] = "y"
 
 sphinx_cmd = [sphinx] + extra_args
 
@@ -23,6 +28,9 @@
 for root, dirs, files in os.walk(src):
 srcfiles.extend([join(root, f) for f in files])
 
+if not os.path.exists(dst):
+os.makedirs(dst)
+
 # run sphinx, putting the html output in a "html" directory
 with open(join(dst, 'sphinx_html.out'), 'w') as out:
 process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')],
@@ -34,7 +42,7 @@
 
 # copy custom CSS file
 css = 'custom.css'
-src_css = join(src, css)
+src_css = join(conf_src, css)
 dst_css = join(dst, 'html', '_static', 'css', css)
 if not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css):
 os.makedirs(os.path.dirname(dst_css), exist_ok=True)
diff --git a/buildtools/get-dts-deps.py b/buildtools/get-dts-deps.py
new file mode 100755
index 00..7114aeb710
--- /dev/null
+++ b/buildtools/get-dts-deps.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement 
of DTS.
+The script returns the standard exit codes in this mode (0 is success, 1 is 
failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS 
pyproject.toml file
+a returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_version_tuple(version_str):
+return tuple(map(int, version_str.split(".")))
+
+
+def _get_dependencies(cfg_file_path):
+cfg = configparser.ConfigParser()
+with open(cfg_file_path) as f:
+dts_deps_file_str = f.read()
+dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+cfg.read_string(dts_deps_file_str)
+
+deps_section = cfg['tool.poetry.dependencies']
+deps = {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+doc_deps_section = cfg['tool.poetry.group.docs.dependencies']
+doc_deps = {dep: doc_deps_section[dep].strip("\"'") for dep in 
doc_deps_section}
+
+return deps | doc_deps
+
+
+def get_missing_imports():
+missing_imports = []
+req_deps = _get_dependencies(_DTS_DEP_FILE_PATH)
+req_dep

Re: [v1,1/5] raw/zxdh: introduce zxdh raw device driver

[Stephen Hemminger]

On Mon,  5 Aug 2024 21:34:53 +0800
Yong Zhang  wrote:

> +struct zxdh_gdma_buff_desc {
> + uint SrcAddr_L;
> + uint DstAddr_L;
> + uint Xpara;
> + uint ZY_para;
> + uint ZY_SrcStep;
> + uint ZY_DstStep;
> + uint ExtAddr;
> + uint LLI_Addr_L;
> + uint LLI_Addr_H;
> + uint ChCont;
> + uint LLI_User;
> + uint ErrAddr;
> + uint Control;
> + uint SrcAddr_H;
> + uint DstAddr_H;
> + uint Reserved;
> +};
> +

Why is this driver using uint type?
It should be using datatypes with explicit size (like uint32_t) to
make sure it can be used on both 32 and 64 bit builds.

[RFC v1 0/2] dts: Ethertype ethdev api test suite

[Nicholas Pratte]

Some general input would be helpful in understanding the validity of
certain test cases within this suite. Each test case is based on the old
DTS test plan. As it currently exists, none of the NICs available for
testing (Intel, Broadcom, Mellanox) properly interface with ethdev api
functions related to tpid changes; some may not support it these
features.

There is a comment on the old DTS test plan related to extended vlans
that needs extra clarification:

"Due to the kernel enables Qinq and cannot be closed, the DPDK only add
`extend on` to make the VLAN filter work normally. Therefore, if the
i40e firmware version >= 8.4 the DPDK can only add `extend on` to make
the VLAN filter work normally"

Capabilities needed related to certain VLAN offloading functionalities,
and these are implemented within the test suite currently.

Some interesting points to highlight in writing this test suite is the
need to modify the tpid at the 'Ether' layer within Scapy, as doing so
modifies the correct tpid needed for testing; this might seem
counter-intuitive at first glance.

Nicholas Pratte (2):
  dts: add additional vlan configuration to testpmd shell class
  dts: port ethertype ethdev api test suite to new dts framework

 dts/framework/config/conf_yaml_schema.json|   3 +-
 dts/framework/remote_session/testpmd_shell.py |  55 +++
 dts/tests/TestSuite_ethertype_config.py   | 381 ++
 3 files changed, 438 insertions(+), 1 deletion(-)
 create mode 100644 dts/tests/TestSuite_ethertype_config.py

-- 
2.44.0

[RFC v1 1/2] dts: add additional vlan configuration to testpmd shell class

[Nicholas Pratte]

The ethertype test suite requires many different internal runtime
vlan offload options to test ethertype configuration. The following patch
 adds a new RxVlanOffloadOptions class that contains all the different
vlan configuration options available within testpmd. Additionally, an
extra method has beena added to adjust both the inner and outer tpids of
ingressing and egressing packets.

Bugzilla ID: 1505

Signed-off-by: Nicholas Pratte 
---
 dts/framework/remote_session/testpmd_shell.py | 55 +++
 1 file changed, 55 insertions(+)

diff --git a/dts/framework/remote_session/testpmd_shell.py 
b/dts/framework/remote_session/testpmd_shell.py
index 288be9a085..268e34f0eb 100644
--- a/dts/framework/remote_session/testpmd_shell.py
+++ b/dts/framework/remote_session/testpmd_shell.py
@@ -70,6 +70,10 @@ class VLANOffloadFlag(Flag):
 #:
 QINQ_STRIP = auto()
 
+def __str__(self):
+"""Return flag name when cast to a string."""
+return self.name
+
 @classmethod
 def from_str_dict(cls, d):
 """Makes an instance from a dict containing the flag member names with 
an "on" value.
@@ -915,6 +919,26 @@ def show_port_stats_all(self) -> list[TestPmdPortStats]:
 iter = re.finditer(r"(^  #*.+#*$[^#]+)^  #*\r$", output, re.MULTILINE)
 return [TestPmdPortStats.parse(block.group(1)) for block in iter]
 
+def set_vlan_tpid(self, port_id: int, tpid: int, inner_id: bool) -> None:
+"""Set ethertype tpid values using the ethdev api.
+
+Args:
+port_id: the ID of the port the tpid is being changed on.
+tpid: The tpid value being changed on the port.
+inner_id: If :data:`True`, set the inner tpid to the specified 
value. If
+:data:`False`, change the outer tpid to the specified value.
+
+Raises:
+InteractiveCommandExecutionError: If either `port_id` or `tpid` 
value is invalid.
+"""
+if tpid < 0 or tpid > 65535:
+raise InteractiveCommandExecutionError("Invalid TPID value given.")
+output = self.send_command(
+f'vlan set {"inner" if inner_id else "outer"} tpid {tpid} 
{port_id}'
+)
+if output.startswith("Invalid port"):
+raise InteractiveCommandExecutionError(f"Invalid port ID {port_id} 
given.")
+
 def show_port_stats(self, port_id: int) -> TestPmdPortStats:
 """Returns the given port statistics.
 
@@ -978,6 +1002,37 @@ def rx_vlan(self, vlan: int, port: int, add: bool, 
verify: bool = True) -> None:
 f"Testpmd failed to {'add' if add else 'remove'} tag 
{vlan} on port {port}."
 )
 
+def set_vlan_offload_option(
+self, port: int, offload_option: VLANOffloadFlag, on: bool, verify: 
bool = True
+) -> None:
+"""Enable extended vlans on the specified port.
+
+Args:
+port: The port number to use, should be within 0-32.
+on: If :data:`True`, extended vlan turn on, otherwise it turns off.
+offload_options: The rx vlan offload option to be set.
+verify: If :data:`True`, the output of the command and show port 
info
+is scanned to verify that vlan stripping was enabled on the 
specified port.
+If not, it is considered an error.
+
+Raises:
+InteractiveCommandExecutionError: If `verify` is :data:`True` and 
stripping
+fails to update.
+"""
+offload_output = self.send_command(
+f"vlan set {str(offload_option).lower()} {'on' if on else 'off'} 
{port}"
+)
+if verify:
+if on ^ (str(offload_option) in 
str(self.show_port_info(port_id=port).vlan_offload)):
+self._logger.debug(
+f"""Failed to set {offload_option} {'on' if on else 'off'} 
port {port}:
+\n{offload_output}
+"""
+)
+raise InteractiveCommandExecutionError(
+f"Testpmd failed to set {offload_option} {'on' if on else 
'off'} port {port}."
+)
+
 def port_stop_all(self, verify: bool = True) -> None:
 """Stop all ports.
 
-- 
2.44.0

[RFC v1 2/2] dts: port ethertype ethdev api test suite to new dts framework

[Nicholas Pratte]

Based off the test cases from the old DTS test suite, the following test
suite assess the behavior of ethertype configuration options found
within the ethdev api.

Bugzilla-ID: 1505

depends-on: 142696 ("dts: add VLAN methods to testpmd shell")
depends-on: 142762 ("dts: add text parser for testpmd verbose output")
depends-on: 139227 ("dts: skip test cases based on capabilities")

Signed-off-by: Nicholas Pratte 
---
 dts/framework/config/conf_yaml_schema.json |   3 +-
 dts/tests/TestSuite_ethertype_config.py| 381 +
 2 files changed, 383 insertions(+), 1 deletion(-)
 create mode 100644 dts/tests/TestSuite_ethertype_config.py

diff --git a/dts/framework/config/conf_yaml_schema.json 
b/dts/framework/config/conf_yaml_schema.json
index f02a310bb5..b5a9bcdf2a 100644
--- a/dts/framework/config/conf_yaml_schema.json
+++ b/dts/framework/config/conf_yaml_schema.json
@@ -187,7 +187,8 @@
   "enum": [
 "hello_world",
 "os_udp",
-"pmd_buffer_scatter"
+"pmd_buffer_scatter",
+"ethertype_config"
   ]
 },
 "test_target": {
diff --git a/dts/tests/TestSuite_ethertype_config.py 
b/dts/tests/TestSuite_ethertype_config.py
new file mode 100644
index 00..4b3e99a081
--- /dev/null
+++ b/dts/tests/TestSuite_ethertype_config.py
@@ -0,0 +1,381 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2023-2024 University of New Hampshire
+
+"""VLAN tpid ethdev api assessment test suite.
+
+Assesses the behavior of Poll Mode Drivers as it relates to their ability to 
adequately interface
+with VLAN tpid functionalities within the ethdev api. Associated test cases 
within this suite may
+check and analyse both verbose output within testpmd and/or forwarding 
behaviors within a paired
+topology. An ethernet device should detect and forward packets as expected 
when using different
+tpids for both standard VLAN and extended QinQ networking. Moreover, if a Poll 
Mode Driver is
+configured with a different tpid, then packets with a corresponding tpid 
should be read properly
+within testpmd verbose output, and packets should be forwarded and not dropped.
+"""
+
+from scapy.layers.inet import IP  # type: ignore[import-untyped]
+from scapy.layers.l2 import Dot1Q, Ether  # type: ignore[import-untyped]
+from scapy.packet import Packet, Raw  # type: ignore[import-untyped]
+
+from framework.params.testpmd import SimpleForwardingModes
+from framework.remote_session.testpmd_shell import (
+NicCapability,
+TestPmdShell,
+VLANOffloadFlag,
+)
+from framework.test_suite import TestSuite, requires
+
+
+class TestEthertypeConfig(TestSuite):
+"""The vlan tpid configuration ethdev test suite for DPDK PMDs."""
+
+def send_packet_and_verify(self, packet: Packet, should_receive: bool) -> 
None:
+"""Sends a specified packet and verify both reception and tpid values.
+
+Sends a specified packet across a paired topology, and if a packet is 
to be received, check
+that said packet is within the list of received packets and verify 
that the tpid is
+unchanged. Otherwise, check that the specified packet was not received.
+
+Args:
+packet: The packet to be sent across the topology.
+should_receive: If :data:`True', ensure that the packet was 
received and the Dot1Q tpid
+is unchanged.
+"""
+received_packets = [
+packets
+for packets in self.send_packet_and_capture(packet)
+if hasattr(packets, "load") and str(packets.load) == "X" * 80
+]
+if should_receive:
+self.verify(len(received_packets) == 1, "Packet should be 
received.")
+self.verify(
+received_packets[0][Ether].type == packet[Ether].type,
+"tpid value changed during transmission",
+)
+else:
+self.verify(not received_packets, "Packet should not be received.")
+
+def verify_verbose_output(
+self, testpmd_shell: TestPmdShell, packet: Packet, expected_output: str
+) -> None:
+"""Send a specified packet and analyse verbose testpmd output for vlan 
information.
+
+Sends a specified packet across a paired topology. Forwarding is not 
checked, but verbose
+output is analysed to ensure that the packet's ptype matches the 
expected output parameter
+which, with respect to this test suite, will either check for standard 
VLAN or QinQ
+tagging information.
+
+Args:
+testpmd_shell: A test case's associated testpmd shell. This is 
needed to stop the
+testpmd shell and gather verbose output for assessment.
+packet: The packet to be sent across the topology.
+expected_output: Output to be checked for within testpmd verbose 
output after a packet
+has been forwarded.
+"""
+self.send_packet_and_capture(packet)
+verbose_output = 
testpmd

[DPDK/testpmd Bug 1514] mlx5 NIC reports checksum unknown when an invalid checksum is present

[bugzilla]

https://bugs.dpdk.org/show_bug.cgi?id=1514

Bug ID: 1514
   Summary: mlx5 NIC reports checksum unknown when an invalid
checksum is present
   Product: DPDK
   Version: unspecified
  Hardware: All
OS: All
Status: UNCONFIRMED
  Severity: normal
  Priority: Normal
 Component: testpmd
  Assignee: dev@dpdk.org
  Reporter: jspew...@iol.unh.edu
  Target Milestone: ---

When running testpmd in verbose mode on a NIC that uses the mlx5_core PMD, if
you send a packet with an invalid IP checksum it will show the ol_flag
`RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN` when it should be
`RTE_MBUF_F_RX_IP_CKSUM_BAD`. According to lib/mbuf/rte_mbuf_core.h
`RTE_MBUF_F_RX_IP_CKSUM_UNKNOWN` should be shown when there is "no information
about the RX IP checksum" and `RTE_MBUF_F_RX_IP_CKSUM_BAD` should be shown when
"the IP checksum in the packet is wrong". This leads to an issue currently
where, from verbose output in testpmd, you cannot distinguish between a packet
having a bad checksum and one not being present at all. It should be noted that
this is also the case for L4 and OUTER_L4 checksums as well.

You can reproduce this by sending a scapy packet with the following structure
to the testpmd shell that is forwarding with verbose mode set to 1:

`pakt = Ether() / IP(chksum=0x0)`

-- 
You are receiving this mail because:
You are the assignee for the bug.

[PATCH] net/iavf: fix core dump when the link is unstable

[Kaiwen Deng]

Physical link instability may cause a core dump.
Unstable physical links can result in a large number of link
change events.Link change events captured by vf before vf
resources are allocated will result in a core dump.

This commit will check if vf_res is invalid before calling it.

Fixes: 5e03e316c753 ("net/iavf: handle virtchnl event message without 
interrupt")
Cc: sta...@dpdk.org

Signed-off-by: Kaiwen Deng 
---
 drivers/net/iavf/iavf_vchnl.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/net/iavf/iavf_vchnl.c b/drivers/net/iavf/iavf_vchnl.c
index 6d5969f084..b85debd40d 100644
--- a/drivers/net/iavf/iavf_vchnl.c
+++ b/drivers/net/iavf/iavf_vchnl.c
@@ -255,7 +255,7 @@ iavf_read_msg_from_pf(struct iavf_adapter *adapter, 
uint16_t buf_len,
case VIRTCHNL_EVENT_LINK_CHANGE:
vf->link_up =
vpe->event_data.link_event.link_status;
-   if (vf->vf_res->vf_cap_flags &
+   if (vf->vf_res != NULL && vf->vf_res->vf_cap_flags &
VIRTCHNL_VF_CAP_ADV_LINK_SPEED) {
vf->link_speed =
vpe->event_data.link_event_adv.link_speed;
-- 
2.34.1

[v2 0/3] L3fwd changes

[Gagandeep Singh]

v2 changes:
*  Handled a comment to enhance the invalid port ID logic
   and added a user option to decide exit or silently skip
   in case invalid port in the rules list.


Gagandeep Singh (3):
  examples/l3fwd: support single route file
  examples/l3fwd: fix return value on rules add
  examples/l3fwd: enhance valid ports checking

 examples/l3fwd/em_route_parse.c  | 33 ++
 examples/l3fwd/l3fwd.h   | 16 +
 examples/l3fwd/l3fwd_em.c| 22 
 examples/l3fwd/l3fwd_fib.c   | 26 +-
 examples/l3fwd/l3fwd_lpm.c   | 26 +-
 examples/l3fwd/l3fwd_route.h |  2 ++
 examples/l3fwd/lpm_route_parse.c | 28 ---
 examples/l3fwd/main.c| 60 ++--
 8 files changed, 154 insertions(+), 59 deletions(-)

-- 
2.25.1

[v2 1/3] examples/l3fwd: support single route file

[Gagandeep Singh]

IPv6 rules file needs to be specified together with IPv4
rules file to configure user given rules. But if user want to
give only IPv4 or only IPv6 rules, application returns error:
"Missing 1 or more rule files"

With this patch application can accept only IPv4,
only IPv6 or both IP rules.

Signed-off-by: Gagandeep Singh 
---
 examples/l3fwd/em_route_parse.c  | 18 ++
 examples/l3fwd/lpm_route_parse.c | 17 ++---
 2 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/examples/l3fwd/em_route_parse.c b/examples/l3fwd/em_route_parse.c
index 6c16832e94..da23356dd6 100644
--- a/examples/l3fwd/em_route_parse.c
+++ b/examples/l3fwd/em_route_parse.c
@@ -249,8 +249,7 @@ void
 read_config_files_em(void)
 {
/* ipv4 check */
-   if (parm_config.rule_ipv4_name != NULL &&
-   parm_config.rule_ipv6_name != NULL) {
+   if (parm_config.rule_ipv4_name != NULL) {
/* ipv4 check */
route_num_v4 = em_add_rules(parm_config.rule_ipv4_name,
&em_route_base_v4, &em_parse_v4_rule);
@@ -258,7 +257,14 @@ read_config_files_em(void)
em_free_routes();
rte_exit(EXIT_FAILURE, "Failed to add EM IPv4 rules\n");
}
-
+   } else {
+   RTE_LOG(INFO, L3FWD, "Missing IPv4 rule file, using default 
instead\n");
+   if (em_add_default_v4_rules() < 0) {
+   em_free_routes();
+   rte_exit(EXIT_FAILURE, "Failed to add default IPv4 
rules\n");
+   }
+   }
+   if (parm_config.rule_ipv6_name != NULL) {
/* ipv6 check */
route_num_v6 = em_add_rules(parm_config.rule_ipv6_name,
&em_route_base_v6, &em_parse_v6_rule);
@@ -267,11 +273,7 @@ read_config_files_em(void)
rte_exit(EXIT_FAILURE, "Failed to add EM IPv6 rules\n");
}
} else {
-   RTE_LOG(INFO, L3FWD, "Missing 1 or more rule files, using 
default instead\n");
-   if (em_add_default_v4_rules() < 0) {
-   em_free_routes();
-   rte_exit(EXIT_FAILURE, "Failed to add default IPv4 
rules\n");
-   }
+   RTE_LOG(INFO, L3FWD, "Missing IPv6 rule file, using default 
instead\n");
if (em_add_default_v6_rules() < 0) {
em_free_routes();
rte_exit(EXIT_FAILURE, "Failed to add default IPv6 
rules\n");
diff --git a/examples/l3fwd/lpm_route_parse.c b/examples/l3fwd/lpm_route_parse.c
index f2028d79e1..f7d44aa2cd 100644
--- a/examples/l3fwd/lpm_route_parse.c
+++ b/examples/l3fwd/lpm_route_parse.c
@@ -271,8 +271,7 @@ lpm_free_routes(void)
 void
 read_config_files_lpm(void)
 {
-   if (parm_config.rule_ipv4_name != NULL &&
-   parm_config.rule_ipv6_name != NULL) {
+   if (parm_config.rule_ipv4_name != NULL) {
/* ipv4 check */
route_num_v4 = lpm_add_rules(parm_config.rule_ipv4_name,
&route_base_v4, &lpm_parse_v4_rule);
@@ -280,7 +279,15 @@ read_config_files_lpm(void)
lpm_free_routes();
rte_exit(EXIT_FAILURE, "Failed to add IPv4 rules\n");
}
+   } else {
+   RTE_LOG(INFO, L3FWD, "Missing IPv4 rule file, using default 
instead\n");
+   if (lpm_add_default_v4_rules() < 0) {
+   lpm_free_routes();
+   rte_exit(EXIT_FAILURE, "Failed to add default IPv4 
rules\n");
+   }
+   }
 
+   if (parm_config.rule_ipv6_name != NULL) {
/* ipv6 check */
route_num_v6 = lpm_add_rules(parm_config.rule_ipv6_name,
&route_base_v6, &lpm_parse_v6_rule);
@@ -289,11 +296,7 @@ read_config_files_lpm(void)
rte_exit(EXIT_FAILURE, "Failed to add IPv6 rules\n");
}
} else {
-   RTE_LOG(INFO, L3FWD, "Missing 1 or more rule files, using 
default instead\n");
-   if (lpm_add_default_v4_rules() < 0) {
-   lpm_free_routes();
-   rte_exit(EXIT_FAILURE, "Failed to add default IPv4 
rules\n");
-   }
+   RTE_LOG(INFO, L3FWD, "Missing IPv6 rule file, using default 
instead\n");
if (lpm_add_default_v6_rules() < 0) {
lpm_free_routes();
rte_exit(EXIT_FAILURE, "Failed to add default IPv6 
rules\n");
-- 
2.25.1

[v2 2/3] examples/l3fwd: fix return value on rules add

[Gagandeep Singh]

fix return value on adding the EM or LPM rules.

Fixes: e7e6dd643092 ("examples/l3fwd: support config file for EM")
Fixes: 52def963fc1c ("examples/l3fwd: support config file for LPM/FIB")
Cc: sean.morris...@intel.com
Cc: sta...@dpdk.org

Signed-off-by: Gagandeep Singh 
---
 examples/l3fwd/em_route_parse.c  | 11 ++-
 examples/l3fwd/lpm_route_parse.c | 11 ++-
 2 files changed, 12 insertions(+), 10 deletions(-)

diff --git a/examples/l3fwd/em_route_parse.c b/examples/l3fwd/em_route_parse.c
index da23356dd6..8b534de5f1 100644
--- a/examples/l3fwd/em_route_parse.c
+++ b/examples/l3fwd/em_route_parse.c
@@ -119,7 +119,7 @@ em_add_rules(const char *rule_path,
char buff[LINE_MAX];
FILE *fh;
unsigned int i = 0, rule_size = sizeof(*next);
-   int val;
+   int val, rc;
 
*proute_base = NULL;
fh = fopen(rule_path, "rb");
@@ -172,13 +172,14 @@ em_add_rules(const char *rule_path,
return -EINVAL;
}
 
-   if (parser(buff + 1, next) != 0) {
+   rc = parser(buff + 1, next);
+   if (rc != 0) {
RTE_LOG(ERR, L3FWD,
-   "%s Line %u: parse rules error\n",
-   rule_path, i);
+   "%s Line %u: parse rules error code = %d\n",
+   rule_path, i, rc);
fclose(fh);
free(route_rules);
-   return -EINVAL;
+   return rc;
}
 
route_cnt++;
diff --git a/examples/l3fwd/lpm_route_parse.c b/examples/l3fwd/lpm_route_parse.c
index f7d44aa2cd..f27b66e838 100644
--- a/examples/l3fwd/lpm_route_parse.c
+++ b/examples/l3fwd/lpm_route_parse.c
@@ -184,7 +184,7 @@ lpm_add_rules(const char *rule_path,
char buff[LINE_MAX];
FILE *fh;
unsigned int i = 0, rule_size = sizeof(*next);
-   int val;
+   int val, rc;
 
*proute_base = NULL;
fh = fopen(rule_path, "rb");
@@ -237,13 +237,14 @@ lpm_add_rules(const char *rule_path,
return -EINVAL;
}
 
-   if (parser(buff + 1, next) != 0) {
+   rc = parser(buff + 1, next);
+   if (rc != 0) {
RTE_LOG(ERR, L3FWD,
-   "%s Line %u: parse rules error\n",
-   rule_path, i);
+   "%s Line %u: parse rules error code = %d\n",
+   rule_path, i, rc);
fclose(fh);
free(route_rules);
-   return -EINVAL;
+   return rc;
}
 
route_cnt++;
-- 
2.25.1

[v2 3/3] examples/l3fwd: enhance valid ports checking

[Gagandeep Singh]

The current port ID validation logic in the routes add code has
two issues:

 - It can pass if port ID in route is 31+.
 - It silently skips rules with disabled or invalid
   port IDs

This patch is:
 - Improving the enabled port IDs check logic.
 - Introducing a user option, "exit_on_failure", to control
   the behavior when attempting to add rules for disabled or
   invalid port IDs (either exit or skip)
 - Creating a port ID validation function for use across
   various setup functions

Signed-off-by: Gagandeep Singh 
---
 examples/l3fwd/em_route_parse.c |  4 +--
 examples/l3fwd/l3fwd.h  | 16 +
 examples/l3fwd/l3fwd_em.c   | 22 
 examples/l3fwd/l3fwd_fib.c  | 26 +-
 examples/l3fwd/l3fwd_lpm.c  | 26 +-
 examples/l3fwd/l3fwd_route.h|  2 ++
 examples/l3fwd/main.c   | 60 +++--
 7 files changed, 122 insertions(+), 34 deletions(-)

diff --git a/examples/l3fwd/em_route_parse.c b/examples/l3fwd/em_route_parse.c
index 8b534de5f1..b80442d7b8 100644
--- a/examples/l3fwd/em_route_parse.c
+++ b/examples/l3fwd/em_route_parse.c
@@ -10,8 +10,8 @@
 #include "l3fwd.h"
 #include "l3fwd_route.h"
 
-static struct em_rule *em_route_base_v4;
-static struct em_rule *em_route_base_v6;
+struct em_rule *em_route_base_v4;
+struct em_rule *em_route_base_v6;
 
 enum {
CB_FLD_DST_ADDR,
diff --git a/examples/l3fwd/l3fwd.h b/examples/l3fwd/l3fwd.h
index 93ce652d02..d6510448a5 100644
--- a/examples/l3fwd/l3fwd.h
+++ b/examples/l3fwd/l3fwd.h
@@ -56,6 +56,15 @@
 #define L3FWD_HASH_ENTRIES (1024*1024*1)
 #endif
 
+/* Select Longest-Prefix, Exact match, Forwarding Information Base or Access 
Control. */
+enum L3FWD_LOOKUP_MODE {
+   L3FWD_LOOKUP_DEFAULT,
+   L3FWD_LOOKUP_LPM,
+   L3FWD_LOOKUP_EM,
+   L3FWD_LOOKUP_FIB,
+   L3FWD_LOOKUP_ACL
+};
+
 struct parm_cfg {
const char *rule_ipv4_name;
const char *rule_ipv6_name;
@@ -101,6 +110,9 @@ extern struct rte_ether_addr 
ports_eth_addr[RTE_MAX_ETHPORTS];
 /* mask of enabled ports */
 extern uint32_t enabled_port_mask;
 
+/* Skip or exit on invalid route */
+extern bool exit_on_failure;
+
 /* Used only in exact match mode. */
 extern int ipv6; /**< ipv6 is false by default. */
 extern uint32_t hash_entry_number;
@@ -218,6 +230,10 @@ init_mem(uint16_t portid, unsigned int nb_mbuf);
 int config_port_max_pkt_len(struct rte_eth_conf *conf,
struct rte_eth_dev_info *dev_info);
 
+int
+l3fwd_validate_routes_port(enum L3FWD_LOOKUP_MODE mode, uint32_t i,
+  bool is_ipv4);
+
 /* Function pointers for ACL, LPM, EM or FIB functionality. */
 void
 setup_acl(const int socketid);
diff --git a/examples/l3fwd/l3fwd_em.c b/examples/l3fwd/l3fwd_em.c
index 31a7e05e39..1a1b48635b 100644
--- a/examples/l3fwd/l3fwd_em.c
+++ b/examples/l3fwd/l3fwd_em.c
@@ -384,9 +384,14 @@ populate_ipv4_flow_into_table(const struct rte_hash *h)
struct in_addr src;
struct in_addr dst;
 
-   if ((1 << em_route_base_v4[i].if_out &
-   enabled_port_mask) == 0)
-   continue;
+   ret = l3fwd_validate_routes_port(L3FWD_LOOKUP_EM, i, true);
+   if (ret) {
+   if (exit_on_failure)
+   rte_exit(EXIT_FAILURE, "IDX: %d: Port ID %d in 
IPv4 rule is not"
+" enabled\n", i, 
em_route_base_v4[i].if_out);
+   else
+   continue;
+   }
 
entry = &em_route_base_v4[i];
convert_ipv4_5tuple(&(entry->v4_key), &newkey);
@@ -436,9 +441,14 @@ populate_ipv6_flow_into_table(const struct rte_hash *h)
struct em_rule *entry;
union ipv6_5tuple_host newkey;
 
-   if ((1 << em_route_base_v6[i].if_out &
-   enabled_port_mask) == 0)
-   continue;
+   ret = l3fwd_validate_routes_port(L3FWD_LOOKUP_EM, i, false);
+   if (ret) {
+   if (exit_on_failure)
+   rte_exit(EXIT_FAILURE, "IDX %d: Port ID %d 
given in IPv6 rule is not"
+" enabled\n", i, 
em_route_base_v6[i].if_out);
+   else
+   continue;
+   }
 
entry = &em_route_base_v6[i];
convert_ipv6_5tuple(&(entry->v6_key), &newkey);
diff --git a/examples/l3fwd/l3fwd_fib.c b/examples/l3fwd/l3fwd_fib.c
index f38b19af3f..9e987e2031 100644
--- a/examples/l3fwd/l3fwd_fib.c
+++ b/examples/l3fwd/l3fwd_fib.c
@@ -670,10 +670,15 @@ setup_fib(const int socketid)
for (i = 0; i < route_num_v4; i++) {
struct in_addr in;
 
-   /* Skip unused ports. */
-   if ((1 << route_base_v4[i].if_out &
-

Re: Portable alternative to inet_ntop?

[Tyler Retzlaff]

On Wed, Jul 24, 2024 at 09:21:13AM -0700, Stephen Hemminger wrote:
> The function inet_ntop is useful to make printable addresses for debugging.
> It is available on Linux and FreeBSD but not on Windows.

i know my reply is late. but the following wasn't usable?
https://learn.microsoft.com/en-us/windows/win32/api/ws2tcpip/nf-ws2tcpip-inet_ntop
it isn't wide (PSTR).

i know trying to use winsock headers is a mess though so if it is hard
duplicating seems fine. i would not be unhappy for more generalized apis
to be built up within dpdk to avoid the platform coupling.

ty

> 
> There are some alternatives:
>   - add yet another OS shim in lib/eal/windows/include.
> Win32 has similar InetNtoP but it uses wide characters.
> 
>   - copy/paste code from FreeBSD into some new functions.
> 
> Hate duplicating code, but portability is a problem here.
> 
> diff --git a/lib/net/rte_ip.h b/lib/net/rte_ip.h
> index 0d103d4127..a9404b4b41 100644
> --- a/lib/net/rte_ip.h
> +++ b/lib/net/rte_ip.h
> @@ -839,6 +839,27 @@ rte_ipv6_get_next_ext(const uint8_t *p, int proto, 
> size_t *ext_len)
> return next_proto;
>  }
>  
> +
> +#define RTE_IPV4_ADDR_FMT_SIZE 16
> +#define RTE_IPV6_ADDR_FMT_SIZE 46
> +
> +__rte_experimental
> +int
> +rte_ipv4_format_addr(char *buf, uint16_t size, const void *addr);
> +
> +__rte_experimental
> +void
> +rte_ipv4_unformat_addr(const char *str, void *addr);
> +
> +__rte_experimental
> +void
> +rte_ipv6_format_addr(char *buf, uint16_t size, const void *addr);
> +
> +__rte_experimental
> +void
> +rte_ipv4_unformat_addr(const char *str, void *addr);

Re: [RFC] ethdev: convert string initialization

[Tyler Retzlaff]

On Thu, Aug 01, 2024 at 02:27:22AM -0700, Ferruh Yigit wrote:
> gcc 15 experimental [1], with -Wextra flag, gives warning in variable
> initialization as string [2].
> 
> The warning has a point when initialized variable is intended to use as
> string, since assignment is missing the required null terminator for
> this case. But warning is useless for our usecase.
> 
> I don't know if this behaviour will change in gcc15, as it is still
> under development. But if not we may need to update our initialization.

i investigated this when msvc had some issues with it as well. we
determined that it is actually iso c standard compliant.

what basically happens is the characters of the string are copied up to
the destination size and any remaining characters from the string
literal (including the NUL) are discarded and that is what leads to
the warning.

yes, the copied bytes lost their NUL terminator but the destination
isn't a string so meh, whatever.

either way i support your fix, it deals with the problem with msvc as
well.

thank you!

> 
> In this patch only updated a few instance to show the issue, there are
> many instances to fix, if we prefer to go this way.
> Other option is to disable warning but it can be useful for actual
> string usecases, so I prefer to keep it.
> 
> [1]
> gcc (GCC) 15.0.0 20240801 (experimental)
> 
> [2]
> ../lib/ethdev/rte_flow.h:906:36:
>   error: initializer-string for array of ???unsigned char??? is too long
> [-Werror=unterminated-string-initialization]
> 906 | .hdr.dst_addr.addr_bytes = "\xff\xff\xff\xff\xff\xff",
> |^~
> 
> ../lib/ethdev/rte_flow.h:907:36:
>   error: initializer-string for array of ???unsigned char??? is too long
>  [-Werror=unterminated-string-initialization]
> 907 | .hdr.src_addr.addr_bytes = "\xff\xff\xff\xff\xff\xff",
> |^~
> 
> ../lib/ethdev/rte_flow.h:1009:25:
>   error: initializer-string for array of ???unsigned char??? is too long
>  [-Werror=unterminated-string-initialization]
> 1009 | "\xff\xff\xff\xff\xff\xff\xff\xff"
>  | ^~
> 
> ../lib/ethdev/rte_flow.h:1012:25:
>   error: initializer-string for array of ???unsigned char??? is too long
>  [-Werror=unterminated-string-initialization]
> 1012 | "\xff\xff\xff\xff\xff\xff\xff\xff"
>  | ^~
> 
> ../lib/ethdev/rte_flow.h:1135:20:
>   error: initializer-string for array of ???unsigned char??? is too long
>  [-Werror=unterminated-string-initialization]
> 1135 | .hdr.vni = "\xff\xff\xff",
>  |^~
> 
> Signed-off-by: Ferruh Yigit 
> ---
>  lib/ethdev/rte_flow.h | 16 +++-
>  1 file changed, 7 insertions(+), 9 deletions(-)
> 
> diff --git a/lib/ethdev/rte_flow.h b/lib/ethdev/rte_flow.h
> index f864578f806b..8b623974cd44 100644
> --- a/lib/ethdev/rte_flow.h
> +++ b/lib/ethdev/rte_flow.h
> @@ -903,8 +903,8 @@ struct rte_flow_item_eth {
>  /** Default mask for RTE_FLOW_ITEM_TYPE_ETH. */
>  #ifndef __cplusplus
>  static const struct rte_flow_item_eth rte_flow_item_eth_mask = {
> - .hdr.dst_addr.addr_bytes = "\xff\xff\xff\xff\xff\xff",
> - .hdr.src_addr.addr_bytes = "\xff\xff\xff\xff\xff\xff",
> + .hdr.dst_addr.addr_bytes = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff },
> + .hdr.src_addr.addr_bytes = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff },
>   .hdr.ether_type = RTE_BE16(0x),
>  };
>  #endif
> @@ -1005,12 +1005,10 @@ struct rte_flow_item_ipv6 {
>  #ifndef __cplusplus
>  static const struct rte_flow_item_ipv6 rte_flow_item_ipv6_mask = {
>   .hdr = {
> - .src_addr =
> - "\xff\xff\xff\xff\xff\xff\xff\xff"
> - "\xff\xff\xff\xff\xff\xff\xff\xff",
> - .dst_addr =
> - "\xff\xff\xff\xff\xff\xff\xff\xff"
> - "\xff\xff\xff\xff\xff\xff\xff\xff",
> + .src_addr = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
> +   0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff },
> + .dst_addr = { 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
> +   0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff },
>   },
>  };
>  #endif
> @@ -1132,7 +1130,7 @@ struct rte_flow_item_vxlan {
>  /** Default mask for RTE_FLOW_ITEM_TYPE_VXLAN. */
>  #ifndef __cplusplus
>  static const struct rte_flow_item_vxlan rte_flow_item_vxlan_mask = {
> - .hdr.vni = "\xff\xff\xff",
> + .hdr.vni = { 0xff, 0xff, 0xff },
>  };
>  #endif
>  
> -- 
> 2.43.0

[PATCH v12 0/5] dts: API docs generation

[Juraj Linkeš]

The generation is done with Sphinx, which DPDK already uses, with
slightly modified configuration of the sidebar present in an if block.

DTS dependencies do not need to be installed, but there is the option to
install doc build dependencies with Poetry:
poetry install --with docs

The build itself may be run with:
meson setup  -Denable_docs=true
ninja -C 

The above will do a full DPDK build with docs. To build just docs:
meson setup 
ninja -C  dts-doc

Python3.10 is required to build the DTS API docs.

The patchset contains the .rst sources which Sphinx uses to generate the
html pages. These were first generated with the sphinx-apidoc utility
and modified to provide a better look. The documentation just doesn't
look that good without the modifications and there isn't enough
configuration options to achieve that without manual changes to the .rst
files. This introduces extra maintenance which involves adding new .rst
files when a new Python module is added or changing the .rst structure
if the Python directory/file structure is changed (moved, renamed
files). This small maintenance burden is outweighed by the flexibility
afforded by the ability to make manual changes to the .rst files.

v10:
Fix dts doc generation issue: Only copy the custom rss file if it exists.

v11:
Added the config option autodoc_mock_imports, which eliminates the need
for DTS dependencies. Added a script that find out which imports need to
be added to autodoc_mock_imports. The script also check the required
Python version for building DTS docs.
Removed tags from the two affected patches which will need to be
reviewed again.

v12:
Added paramiko to the required dependencies of get-dts-deps.py.

Juraj Linkeš (5):
  dts: update params and parser docstrings
  dts: add doc generation dependencies
  dts: add API doc sources
  doc: meson doc API build dir variable
  dts: add API doc generation

 buildtools/call-sphinx-build.py   |  10 +-
 buildtools/get-dts-deps.py|  78 +++
 buildtools/meson.build|   1 +
 doc/api/doxy-api-index.md |   3 +
 doc/api/doxy-api.conf.in  |   2 +
 doc/api/meson.build   |   8 +-
 doc/guides/conf.py|  41 +-
 doc/guides/contributing/documentation.rst |   2 +
 doc/guides/contributing/patches.rst   |   4 +
 doc/guides/meson.build|   1 +
 doc/guides/tools/dts.rst  |  39 +-
 dts/doc/conf_yaml_schema.json |   1 +
 dts/doc/framework.config.rst  |  12 +
 dts/doc/framework.config.types.rst|   6 +
 dts/doc/framework.exception.rst   |   6 +
 dts/doc/framework.logger.rst  |   6 +
 dts/doc/framework.params.eal.rst  |   6 +
 dts/doc/framework.params.rst  |  14 +
 dts/doc/framework.params.testpmd.rst  |   6 +
 dts/doc/framework.params.types.rst|   6 +
 dts/doc/framework.parser.rst  |   6 +
 .../framework.remote_session.dpdk_shell.rst   |   6 +
 ...ote_session.interactive_remote_session.rst |   6 +
 ...ework.remote_session.interactive_shell.rst |   6 +
 .../framework.remote_session.python_shell.rst |   6 +
 ...ramework.remote_session.remote_session.rst |   6 +
 dts/doc/framework.remote_session.rst  |  18 +
 .../framework.remote_session.ssh_session.rst  |   6 +
 ...framework.remote_session.testpmd_shell.rst |   6 +
 dts/doc/framework.runner.rst  |   6 +
 dts/doc/framework.settings.rst|   6 +
 dts/doc/framework.test_result.rst |   6 +
 dts/doc/framework.test_suite.rst  |   6 +
 dts/doc/framework.testbed_model.cpu.rst   |   6 +
 .../framework.testbed_model.linux_session.rst |   6 +
 dts/doc/framework.testbed_model.node.rst  |   6 +
 .../framework.testbed_model.os_session.rst|   6 +
 dts/doc/framework.testbed_model.port.rst  |   6 +
 .../framework.testbed_model.posix_session.rst |   6 +
 dts/doc/framework.testbed_model.rst   |  26 +
 dts/doc/framework.testbed_model.sut_node.rst  |   6 +
 dts/doc/framework.testbed_model.tg_node.rst   |   6 +
 ..._generator.capturing_traffic_generator.rst |   6 +
 ...mework.testbed_model.traffic_generator.rst |  14 +
 testbed_model.traffic_generator.scapy.rst |   6 +
 ...el.traffic_generator.traffic_generator.rst |   6 +
 ...framework.testbed_model.virtual_device.rst |   6 +
 dts/doc/framework.utils.rst   |   6 +
 dts/doc/index.rst |  43 ++
 dts/doc/meson.build   |  30 +
 dts/framework/params/__init__.py  |   4 +-
 dts/framework/params/eal.py   |   7 +-
 dts/framework/params/types.py |   3 +-
 dts/framework/parser.py   |   4 +-
 dts/meson.build   |  15 +
 dts/poetry.lock   | 521 

[PATCH v12 1/5] dts: update params and parser docstrings

[Juraj Linkeš]

Address a few errors reported by Sphinx when generating documentation:
framework/params/__init__.py:docstring of framework.params.modify_str:3:
WARNING: Inline interpreted text or phrase reference start-string
without end-string.
framework/params/eal.py:docstring of framework.params.eal.EalParams:35:
WARNING: Definition list ends without a blank line; unexpected
unindent.
framework/params/types.py:docstring of framework.params.types:8:
WARNING: Inline strong start-string without end-string.
framework/params/types.py:docstring of framework.params.types:9:
WARNING: Inline strong start-string without end-string.
framework/parser.py:docstring of framework.parser.TextParser:33: ERROR:
Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser:43: ERROR:
Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser:49: ERROR:
Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser.wrap:8:
ERROR: Unexpected indentation.
framework/parser.py:docstring of framework.parser.TextParser.wrap:9:
WARNING: Block quote ends without a blank line; unexpected unindent.

Fixes: 87ba4cdc0dbb ("dts: use Unpack for type checking and hinting")
Fixes: d70159cb62f5 ("dts: add params manipulation module")
Fixes: 967fc62b0a43 ("dts: refactor EAL parameters class")
Fixes: 818fe14e3422 ("dts: add parsing utility module")
Cc: luca.vizza...@arm.com

Signed-off-by: Juraj Linkeš 
Reviewed-by: Luca Vizzarro 
---
 dts/framework/params/__init__.py | 4 ++--
 dts/framework/params/eal.py  | 7 +--
 dts/framework/params/types.py| 3 ++-
 dts/framework/parser.py  | 4 ++--
 4 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/dts/framework/params/__init__.py b/dts/framework/params/__init__.py
index 5a6fd93053..1ae227d7b4 100644
--- a/dts/framework/params/__init__.py
+++ b/dts/framework/params/__init__.py
@@ -53,9 +53,9 @@ def reduced_fn(value):
 
 
 def modify_str(*funcs: FnPtr) -> Callable[[T], T]:
-"""Class decorator modifying the ``__str__`` method with a function 
created from its arguments.
+r"""Class decorator modifying the ``__str__`` method with a function 
created from its arguments.
 
-The :attr:`FnPtr`s fed to the decorator are executed from left to right in 
the arguments list
+The :attr:`FnPtr`\s fed to the decorator are executed from left to right 
in the arguments list
 order.
 
 Args:
diff --git a/dts/framework/params/eal.py b/dts/framework/params/eal.py
index 8d7766fefc..cf1594353a 100644
--- a/dts/framework/params/eal.py
+++ b/dts/framework/params/eal.py
@@ -26,13 +26,16 @@ class EalParams(Params):
 prefix: Set the file prefix string with which to start DPDK, e.g.: 
``prefix="vf"``.
 no_pci: Switch to disable PCI bus, e.g.: ``no_pci=True``.
 vdevs: Virtual devices, e.g.::
+
 vdevs=[
 VirtualDevice('net_ring0'),
 VirtualDevice('net_ring1')
 ]
+
 ports: The list of ports to allow.
-other_eal_param: user defined DPDK EAL parameters, e.g.:
-``other_eal_param='--single-file-segments'``
+other_eal_param: user defined DPDK EAL parameters, e.g.::
+
+``other_eal_param='--single-file-segments'``
 """
 
 lcore_list: LogicalCoreList | None = field(default=None, 
metadata=Params.short("l"))
diff --git a/dts/framework/params/types.py b/dts/framework/params/types.py
index e668f658d8..d77c4625fb 100644
--- a/dts/framework/params/types.py
+++ b/dts/framework/params/types.py
@@ -6,7 +6,8 @@
 TypedDicts can be used in conjunction with Unpack and kwargs for type hinting 
on function calls.
 
 Example:
-..code:: python
+.. code:: python
+
 def create_testpmd(**kwargs: Unpack[TestPmdParamsDict]):
 params = TestPmdParams(**kwargs)
 """
diff --git a/dts/framework/parser.py b/dts/framework/parser.py
index 741dfff821..7254c75b71 100644
--- a/dts/framework/parser.py
+++ b/dts/framework/parser.py
@@ -46,7 +46,7 @@ class TextParser(ABC):
 Example:
 The following example makes use of and demonstrates every parser 
function available:
 
-..code:: python
+.. code:: python
 
 from dataclasses import dataclass, field
 from enum import Enum
@@ -90,7 +90,7 @@ def wrap(parser_fn: ParserFn, wrapper_fn: Callable) -> 
ParserFn:
 """Makes a wrapped parser function.
 
 `parser_fn` is called and if a non-None value is returned, 
`wrapper_function` is called with
-it. Otherwise the function returns early with None. In pseudo-code:
+it. Otherwise the function returns early with None. In pseudo-code::
 
 intermediate_value := parser_fn(input)
 if intermediary_value is None then
-- 
2.34.1

[PATCH v12 2/5] dts: add doc generation dependencies

[Juraj Linkeš]

Sphinx imports every Python module (through the autodoc extension)
when generating documentation from docstrings, meaning all DTS
dependencies, including Python version, should be satisfied. This is not
a hard requirement, as imports from dependencies may be mocked in the
autodoc_mock_imports autodoc option.
In case DTS developers want to use a Sphinx installation from their
virtualenv, we provide an optional Poetry group for doc generation. The
pyelftools package is there so that meson picks up the correct Python
installation, as pyelftools is required by the build system.

Signed-off-by: Juraj Linkeš 
---
 dts/poetry.lock| 521 +++--
 dts/pyproject.toml |   8 +
 2 files changed, 517 insertions(+), 12 deletions(-)

diff --git a/dts/poetry.lock b/dts/poetry.lock
index 5f8fa03933..2dd8bad498 100644
--- a/dts/poetry.lock
+++ b/dts/poetry.lock
@@ -1,5 +1,16 @@
 # This file is automatically @generated by Poetry 1.8.2 and should not be 
changed by hand.
 
+[[package]]
+name = "alabaster"
+version = "0.7.13"
+description = "A configurable sidebar-enabled Sphinx theme"
+optional = false
+python-versions = ">=3.6"
+files = [
+{file = "alabaster-0.7.13-py3-none-any.whl", hash = 
"sha256:1ee19aca801bbabb5ba3f5f258e4422dfa86f82f3e9cefb0859b283cdd7f62a3"},
+{file = "alabaster-0.7.13.tar.gz", hash = 
"sha256:a27a4a084d5e690e16e01e03ad2b2e552c61a65469419b907243193de1a84ae2"},
+]
+
 [[package]]
 name = "attrs"
 version = "23.1.0"
@@ -18,6 +29,23 @@ docs = ["furo", "myst-parser", "sphinx", 
"sphinx-notfound-page", "sphinxcontrib-
 tests = ["attrs[tests-no-zope]", "zope-interface"]
 tests-no-zope = ["cloudpickle", "hypothesis", "mypy (>=1.1.1)", "pympler", 
"pytest (>=4.3.0)", "pytest-mypy-plugins", "pytest-xdist[psutil]"]
 
+[[package]]
+name = "babel"
+version = "2.13.1"
+description = "Internationalization utilities"
+optional = false
+python-versions = ">=3.7"
+files = [
+{file = "Babel-2.13.1-py3-none-any.whl", hash = 
"sha256:7077a4984b02b6727ac10f1f7294484f737443d7e2e66c5e4380e41a3ae0b4ed"},
+{file = "Babel-2.13.1.tar.gz", hash = 
"sha256:33e0952d7dd6374af8dbf6768cc4ddf3ccfefc244f9986d4074704f2fbd18900"},
+]
+
+[package.dependencies]
+setuptools = {version = "*", markers = "python_version >= \"3.12\""}
+
+[package.extras]
+dev = ["freezegun (>=1.0,<2.0)", "pytest (>=6.0)", "pytest-cov"]
+
 [[package]]
 name = "bcrypt"
 version = "4.0.1"
@@ -86,6 +114,17 @@ d = ["aiohttp (>=3.7.4)"]
 jupyter = ["ipython (>=7.8.0)", "tokenize-rt (>=3.2.0)"]
 uvloop = ["uvloop (>=0.15.2)"]
 
+[[package]]
+name = "certifi"
+version = "2023.7.22"
+description = "Python package for providing Mozilla's CA Bundle."
+optional = false
+python-versions = ">=3.6"
+files = [
+{file = "certifi-2023.7.22-py3-none-any.whl", hash = 
"sha256:92d6037539857d8206b8f6ae472e8b77db8058fec5937a1ef3f54304089edbb9"},
+{file = "certifi-2023.7.22.tar.gz", hash = 
"sha256:539cc1d13202e33ca466e88b2807e29f4c13049d6d87031a3c110744495cb082"},
+]
+
 [[package]]
 name = "cffi"
 version = "1.15.1"
@@ -162,6 +201,105 @@ files = [
 [package.dependencies]
 pycparser = "*"
 
+[[package]]
+name = "charset-normalizer"
+version = "3.3.2"
+description = "The Real First Universal Charset Detector. Open, modern and 
actively maintained alternative to Chardet."
+optional = false
+python-versions = ">=3.7.0"
+files = [
+{file = "charset-normalizer-3.3.2.tar.gz", hash = 
"sha256:f30c3cb33b24454a82faecaf01b19c18562b1e89558fb6c56de4d9118a032fd5"},
+{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_universal2.whl", 
hash = 
"sha256:25baf083bf6f6b341f4121c2f3c548875ee6f5339300e08be3f2b2ba1721cdd3"},
+{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_x86_64.whl", 
hash = 
"sha256:06435b539f889b1f6f4ac1758871aae42dc3a8c0e24ac9e60c2384973ad73027"},
+{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl", hash 
= "sha256:9063e24fdb1e498ab71cb7419e24622516c4a04476b17a2dab57e8baa30d6e03"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl",
 hash = 
"sha256:6897af51655e3691ff853668779c7bad41579facacf5fd7253b0133308cf000d"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl",
 hash = 
"sha256:1d3193f4a680c64b4b6a9115943538edb896edc190f0b222e73761716519268e"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl",
 hash = 
"sha256:cd70574b12bb8a4d2aaa0094515df2463cb429d8536cfb6c7ce983246983e5a6"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl",
 hash = 
"sha256:8465322196c8b4d7ab6d1e049e4c5cb460d0394da4a27d23cc242fbf0034b6b5"},
+{file = 
"charset_normalizer-3.3.2-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl",
 hash = 
"sha256:a9a8e9031d613fd2009c182b69c7b2c1ef8239a0efb1df3f7c8da66d5dd3d537"},
+{file = "charset_normalizer-3.3.2-cp310-cp

[PATCH v12 3/5] dts: add API doc sources

[Juraj Linkeš]

These sources could be generated with the sphinx-apidoc utility, but
that doesn't give us enough flexibility, such as sorting the order of
modules or changing the headers of the modules.

The sources included in this patch were in fact generated by said
utility, but modified to improve the look of the documentation. The
improvements are mainly in toctree definitions and the titles of the
modules/packages. These were made with specific Sphinx config options in
mind.

Signed-off-by: Juraj Linkeš 
Reviewed-by: Luca Vizzarro 
Reviewed-by: Jeremy Spewock 
Tested-by: Luca Vizzarro 
---
 dts/doc/conf_yaml_schema.json |  1 +
 dts/doc/framework.config.rst  | 12 ++
 dts/doc/framework.config.types.rst|  6 +++
 dts/doc/framework.exception.rst   |  6 +++
 dts/doc/framework.logger.rst  |  6 +++
 dts/doc/framework.params.eal.rst  |  6 +++
 dts/doc/framework.params.rst  | 14 ++
 dts/doc/framework.params.testpmd.rst  |  6 +++
 dts/doc/framework.params.types.rst|  6 +++
 dts/doc/framework.parser.rst  |  6 +++
 .../framework.remote_session.dpdk_shell.rst   |  6 +++
 ...ote_session.interactive_remote_session.rst |  6 +++
 ...ework.remote_session.interactive_shell.rst |  6 +++
 .../framework.remote_session.python_shell.rst |  6 +++
 ...ramework.remote_session.remote_session.rst |  6 +++
 dts/doc/framework.remote_session.rst  | 18 
 .../framework.remote_session.ssh_session.rst  |  6 +++
 ...framework.remote_session.testpmd_shell.rst |  6 +++
 dts/doc/framework.runner.rst  |  6 +++
 dts/doc/framework.settings.rst|  6 +++
 dts/doc/framework.test_result.rst |  6 +++
 dts/doc/framework.test_suite.rst  |  6 +++
 dts/doc/framework.testbed_model.cpu.rst   |  6 +++
 .../framework.testbed_model.linux_session.rst |  6 +++
 dts/doc/framework.testbed_model.node.rst  |  6 +++
 .../framework.testbed_model.os_session.rst|  6 +++
 dts/doc/framework.testbed_model.port.rst  |  6 +++
 .../framework.testbed_model.posix_session.rst |  6 +++
 dts/doc/framework.testbed_model.rst   | 26 +++
 dts/doc/framework.testbed_model.sut_node.rst  |  6 +++
 dts/doc/framework.testbed_model.tg_node.rst   |  6 +++
 ..._generator.capturing_traffic_generator.rst |  6 +++
 ...mework.testbed_model.traffic_generator.rst | 14 ++
 testbed_model.traffic_generator.scapy.rst |  6 +++
 ...el.traffic_generator.traffic_generator.rst |  6 +++
 ...framework.testbed_model.virtual_device.rst |  6 +++
 dts/doc/framework.utils.rst   |  6 +++
 dts/doc/index.rst | 43 +++
 38 files changed, 314 insertions(+)
 create mode 12 dts/doc/conf_yaml_schema.json
 create mode 100644 dts/doc/framework.config.rst
 create mode 100644 dts/doc/framework.config.types.rst
 create mode 100644 dts/doc/framework.exception.rst
 create mode 100644 dts/doc/framework.logger.rst
 create mode 100644 dts/doc/framework.params.eal.rst
 create mode 100644 dts/doc/framework.params.rst
 create mode 100644 dts/doc/framework.params.testpmd.rst
 create mode 100644 dts/doc/framework.params.types.rst
 create mode 100644 dts/doc/framework.parser.rst
 create mode 100644 dts/doc/framework.remote_session.dpdk_shell.rst
 create mode 100644 
dts/doc/framework.remote_session.interactive_remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.interactive_shell.rst
 create mode 100644 dts/doc/framework.remote_session.python_shell.rst
 create mode 100644 dts/doc/framework.remote_session.remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.rst
 create mode 100644 dts/doc/framework.remote_session.ssh_session.rst
 create mode 100644 dts/doc/framework.remote_session.testpmd_shell.rst
 create mode 100644 dts/doc/framework.runner.rst
 create mode 100644 dts/doc/framework.settings.rst
 create mode 100644 dts/doc/framework.test_result.rst
 create mode 100644 dts/doc/framework.test_suite.rst
 create mode 100644 dts/doc/framework.testbed_model.cpu.rst
 create mode 100644 dts/doc/framework.testbed_model.linux_session.rst
 create mode 100644 dts/doc/framework.testbed_model.node.rst
 create mode 100644 dts/doc/framework.testbed_model.os_session.rst
 create mode 100644 dts/doc/framework.testbed_model.port.rst
 create mode 100644 dts/doc/framework.testbed_model.posix_session.rst
 create mode 100644 dts/doc/framework.testbed_model.rst
 create mode 100644 dts/doc/framework.testbed_model.sut_node.rst
 create mode 100644 dts/doc/framework.testbed_model.tg_node.rst
 create mode 100644 
dts/doc/framework.testbed_model.traffic_generator.capturing_traffic_generator.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.rst
 create mode 100644 dts/doc/framework.testbed_model.traffic_generator.scapy.rst
 create mode 100644 
dts/doc/framework.testbed_model.traffic_generator.traffic_generator

[PATCH v12 4/5] doc: meson doc API build dir variable

[Juraj Linkeš]

The three instances of the path 'dpdk_build_root/doc/api' are replaced
with a variable, moving the definition to one place.

Signed-off-by: Juraj Linkeš 
Reviewed-by: Luca Vizzarro 
Reviewed-by: Jeremy Spewock 
Acked-by: Bruce Richardson 
Tested-by: Luca Vizzarro 
Tested-by: Nicholas Pratte 
---
 doc/api/meson.build | 7 ---
 1 file changed, 4 insertions(+), 3 deletions(-)

diff --git a/doc/api/meson.build b/doc/api/meson.build
index 5b50692df9..b828b1ed66 100644
--- a/doc/api/meson.build
+++ b/doc/api/meson.build
@@ -1,6 +1,7 @@
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Luca Boccassi 
 
+doc_api_build_dir = meson.current_build_dir()
 doxygen = find_program('doxygen', required: get_option('enable_docs'))
 
 if not doxygen.found()
@@ -32,10 +33,10 @@ example = custom_target('examples.dox',
 # set up common Doxygen configuration
 cdata = configuration_data()
 cdata.set('VERSION', meson.project_version())
-cdata.set('API_EXAMPLES', join_paths(dpdk_build_root, 'doc', 'api', 
'examples.dox'))
-cdata.set('OUTPUT', join_paths(dpdk_build_root, 'doc', 'api'))
+cdata.set('API_EXAMPLES', join_paths(doc_api_build_dir, 'examples.dox'))
+cdata.set('OUTPUT', doc_api_build_dir)
 cdata.set('TOPDIR', dpdk_source_root)
-cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, 
join_paths(dpdk_build_root, 'doc', 'api')]))
+cdata.set('STRIP_FROM_PATH', ' '.join([dpdk_source_root, doc_api_build_dir]))
 cdata.set('WARN_AS_ERROR', 'NO')
 if get_option('werror')
 cdata.set('WARN_AS_ERROR', 'YES')
-- 
2.34.1

[PATCH v12 5/5] dts: add API doc generation

[Juraj Linkeš]

The tool used to generate DTS API docs is Sphinx, which is already in
use in DPDK. The same configuration is used to preserve style with one
DTS-specific configuration (so that the DPDK docs are unchanged) that
modifies how the sidebar displays the content.

Sphinx generates the documentation from Python docstrings. The docstring
format is the Google format [0] which requires the sphinx.ext.napoleon
extension. The other extension, sphinx.ext.intersphinx, enables linking
to objects in external documentations, such as the Python documentation.

There is one requirement for building DTS docs - the same Python version
as DTS or higher, because Sphinx's autodoc extension imports the code.

The dependencies needed to import the code don't have to be satisfied,
as the autodoc extension allows us to mock the imports. The missing
packages are taken from the DTS pyproject.toml file.

[0] https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings

Signed-off-by: Juraj Linkeš 
---
 buildtools/call-sphinx-build.py   | 10 ++-
 buildtools/get-dts-deps.py| 78 +++
 buildtools/meson.build|  1 +
 doc/api/doxy-api-index.md |  3 +
 doc/api/doxy-api.conf.in  |  2 +
 doc/api/meson.build   |  1 +
 doc/guides/conf.py| 41 +++-
 doc/guides/contributing/documentation.rst |  2 +
 doc/guides/contributing/patches.rst   |  4 ++
 doc/guides/meson.build|  1 +
 doc/guides/tools/dts.rst  | 39 +++-
 dts/doc/meson.build   | 30 +
 dts/meson.build   | 15 +
 meson.build   |  1 +
 14 files changed, 225 insertions(+), 3 deletions(-)
 create mode 100755 buildtools/get-dts-deps.py
 create mode 100644 dts/doc/meson.build
 create mode 100644 dts/meson.build

diff --git a/buildtools/call-sphinx-build.py b/buildtools/call-sphinx-build.py
index 623e7363ee..5dd59907cd 100755
--- a/buildtools/call-sphinx-build.py
+++ b/buildtools/call-sphinx-build.py
@@ -15,6 +15,11 @@
 
 # set the version in environment for sphinx to pick up
 os.environ['DPDK_VERSION'] = version
+conf_src = src
+if src.find('dts') != -1:
+if '-c' in extra_args:
+conf_src = extra_args[extra_args.index('-c') + 1]
+os.environ['DTS_BUILD'] = "y"
 
 sphinx_cmd = [sphinx] + extra_args
 
@@ -23,6 +28,9 @@
 for root, dirs, files in os.walk(src):
 srcfiles.extend([join(root, f) for f in files])
 
+if not os.path.exists(dst):
+os.makedirs(dst)
+
 # run sphinx, putting the html output in a "html" directory
 with open(join(dst, 'sphinx_html.out'), 'w') as out:
 process = run(sphinx_cmd + ['-b', 'html', src, join(dst, 'html')],
@@ -34,7 +42,7 @@
 
 # copy custom CSS file
 css = 'custom.css'
-src_css = join(src, css)
+src_css = join(conf_src, css)
 dst_css = join(dst, 'html', '_static', 'css', css)
 if not os.path.exists(dst_css) or not filecmp.cmp(src_css, dst_css):
 os.makedirs(os.path.dirname(dst_css), exist_ok=True)
diff --git a/buildtools/get-dts-deps.py b/buildtools/get-dts-deps.py
new file mode 100755
index 00..309b83cb5c
--- /dev/null
+++ b/buildtools/get-dts-deps.py
@@ -0,0 +1,78 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 PANTHEON.tech s.r.o.
+#
+
+"""Utilities for DTS dependencies.
+
+The module can be used as an executable script,
+which verifies that the running Python version meets the version requirement 
of DTS.
+The script returns the standard exit codes in this mode (0 is success, 1 is 
failure).
+
+The module also contains a function, get_missing_imports,
+which looks for runtime and doc generation dependencies in the DTS 
pyproject.toml file
+a returns a list of module names used in an import statement that are missing.
+"""
+
+import configparser
+import importlib.metadata
+import importlib.util
+import os.path
+import platform
+
+_VERSION_COMPARISON_CHARS = '^<>='
+_EXTRA_DEPS = {'invoke': '>=1.3', 'paramiko': '>=2.4'}
+_DPDK_ROOT = os.path.dirname(os.path.dirname(__file__))
+_DTS_DEP_FILE_PATH = os.path.join(_DPDK_ROOT, 'dts', 'pyproject.toml')
+
+
+def _get_version_tuple(version_str):
+return tuple(map(int, version_str.split(".")))
+
+
+def _get_dependencies(cfg_file_path):
+cfg = configparser.ConfigParser()
+with open(cfg_file_path) as f:
+dts_deps_file_str = f.read()
+dts_deps_file_str = dts_deps_file_str.replace("\n]", "]")
+cfg.read_string(dts_deps_file_str)
+
+deps_section = cfg['tool.poetry.dependencies']
+deps = {dep: deps_section[dep].strip('"\'') for dep in deps_section}
+doc_deps_section = cfg['tool.poetry.group.docs.dependencies']
+doc_deps = {dep: doc_deps_section[dep].strip("\"'") for dep in 
doc_deps_section}
+
+return deps | doc_deps
+
+
+def get_missing_imports():
+missing_imports = []
+req_deps = _get_dependencies(_DTS_DEP_FI