Re: [dpdk-dev] [PATCH v2] usertools/dpdk-devbind: add support for PCI wildcards

[Burakov, Anatoly]

On 20-Aug-20 4:43 PM, Bruce Richardson wrote:

When binding or unbinding a range of devices, it can be useful to use
wildcards to specify the devices rather than repeating the same prefix
multiple times. We can use the python "glob" module to give us this
functionality - at least for PCI devices - by checking /sys for matching
files.

Examples of use from my system:

 ./dpdk-devbind.py -b vfio-pci 80:04.*
 ./dpdk-devbind.py -u 80:04.[2-7]

The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The
second then unbinds six of those devices, 80:04.2..80:04.7, from any
driver.

Signed-off-by: Bruce Richardson 
Tested-by: Ferruh Yigit 
---
V2: added help text additions
---
  usertools/dpdk-devbind.py | 16 
  1 file changed, 16 insertions(+)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 86b6b53c40..d13defbe1a 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -8,6 +8,7 @@
  import os
  import getopt
  import subprocess
+from glob import glob
  from os.path import exists, abspath, dirname, basename
  
  if sys.version_info.major < 3:

@@ -89,6 +90,8 @@ def usage():
  where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" 
syntax
  or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may
  also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc.
+If devices are specified using PCI bus:device:func format, then
+shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3]
  
  Options:

  --help, --usage:
@@ -145,6 +148,9 @@ def usage():
  To bind :02:00.0 and :02:00.1 to the ixgbe kernel driver
  %(argv0)s -b ixgbe 02:00.0 02:00.1
  
+To bind all funcions on device :02:00 to ixgbe kernel driver

+%(argv0)s -b ixgbe 02:00.*
+
  """ % locals())  # replace items from local variables
  
  
@@ -689,6 +695,16 @@ def parse_args():

  else:
  b_flag = arg
  
+# resolve any PCI globs in the args

+new_args = []
+sysfs_path = "/sys/bus/pci/devices/"
+for arg in args:
+globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + ":" + arg)


os.path.join()?


+if globbed_arg:
+new_args.extend([a[len(sysfs_path):] for a in globbed_arg])


os.path.basename()?


+else:
+new_args.append(arg)
+args = new_args
  
  def do_arg_actions():

  '''do the actual action requested by the user'''




--
Thanks,
Anatoly

[dpdk-dev] [PATCH v5 1/9] usertools/dpdk-telemetry-client: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Cc: Kevin Laatz 

Signed-off-by: Louise Kilheeney 
Acked-by: Bruce Richardson 
---
 usertools/dpdk-telemetry-client.py | 20 
 1 file changed, 4 insertions(+), 16 deletions(-)

diff --git a/usertools/dpdk-telemetry-client.py 
b/usertools/dpdk-telemetry-client.py
index 98d28fa89b..fa599046a4 100755
--- a/usertools/dpdk-telemetry-client.py
+++ b/usertools/dpdk-telemetry-client.py
@@ -1,10 +1,7 @@
-#! /usr/bin/env python
+#! /usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2018 Intel Corporation
 
-from __future__ import print_function
-from __future__ import unicode_literals
-
 import socket
 import os
 import sys
@@ -18,15 +15,6 @@
 GLOBAL_METRICS_REQ = 
"{\"action\":0,\"command\":\"global_stat_values\",\"data\":null}"
 DEFAULT_FP = "/var/run/dpdk/default_client"
 
-try:
-raw_input  # Python 2
-except NameError:
-raw_input = input  # Python 3
-
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not work 
in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
-
 class Socket:
 
 def __init__(self):
@@ -53,7 +41,7 @@ def __init__(self): # Creates a client instance
 def __del__(self):
 try:
 if self.unregistered == 0:
-self.unregister();
+self.unregister()
 except:
 print("Error - Client could not be destroyed")
 
@@ -86,7 +74,7 @@ def requestMetrics(self): # Requests metrics for given client
 
 def repeatedlyRequestMetrics(self, sleep_time): # Recursively requests 
metrics for given client
 print("\nPlease enter the number of times you'd like to continuously 
request Metrics:")
-n_requests = int(raw_input("\n:"))
+n_requests = int(input("\n:"))
 print("\033[F") #Removes the user input from screen, cleans it up
 print("\033[K")
 for i in range(n_requests):
@@ -107,7 +95,7 @@ def interactiveMenu(self, sleep_time): # Creates Interactive 
menu within the scr
 print("[4] Unregister client")
 
 try:
-self.choice = int(raw_input("\n:"))
+self.choice = int(input("\n:"))
 print("\033[F") #Removes the user input for screen, cleans it 
up
 print("\033[K")
 if self.choice == 1:
-- 
2.17.1

[dpdk-dev] [PATCH v5 0/9]adding support for python 3 only.

[Louise Kilheeney]

This patch set converts all python scripts in the project to use
python3 only and removes all deprecation notices associated with these 
changes. This is due to python2 being EOL in January 2020.

Louise Kilheeney (9):
  usertools/dpdk-telemetry-client: support python3 only
  usertools/dpdk-devbind: support python3 only
  usertools/dpdk-pmdinfo: support python3 only
  usertools/cpu_layout: support python3 only
  app/test-cmdline: support python3 only
  app/test: support python3 only
  devtools: support python3 only
  config/arm: support python3 only
  app/test-bbdev: support python3 only

 app/test-bbdev/test-bbdev.py  |  7 +--
 app/test-cmdline/cmdline_test.py  |  9 ++---
 app/test-cmdline/cmdline_test_data.py |  1 +
 app/test/autotest.py  |  7 +--
 app/test/autotest_data.py |  1 +
 app/test/autotest_runner.py   | 21 -
 app/test/autotest_test_funcs.py   |  1 +
 config/arm/armv8_machine.py   |  2 +-
 devtools/update_version_map_abi.py|  7 +--
 mk/rte.sdktest.mk |  6 +++---
 usertools/cpu_layout.py   | 13 ++---
 usertools/dpdk-devbind.py | 22 --
 usertools/dpdk-pmdinfo.py |  7 +--
 usertools/dpdk-telemetry-client.py| 20 
 14 files changed, 31 insertions(+), 93 deletions(-)

-- 
2.17.1

[dpdk-dev] [PATCH v5 3/9] usertools/dpdk-pmdinfo: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Cc: Neil Horman 

Signed-off-by: Louise Kilheeney 
Reviewed-by: Bruce Richardson 
---
 usertools/dpdk-pmdinfo.py | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/usertools/dpdk-pmdinfo.py b/usertools/dpdk-pmdinfo.py
index f9ed755176..1661982791 100755
--- a/usertools/dpdk-pmdinfo.py
+++ b/usertools/dpdk-pmdinfo.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2016  Neil Horman 
 
@@ -7,8 +7,6 @@
 # Utility to dump PMD_INFO_STRING support from an object file
 #
 # -
-from __future__ import print_function
-from __future__ import unicode_literals
 import json
 import io
 import os
@@ -28,9 +26,6 @@
 pcidb = None
 
 # ===
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not 
work in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
 
 class Vendor:
 """
-- 
2.17.1

[dpdk-dev] [PATCH v5 4/9] usertools/cpu_layout: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Signed-off-by: Louise Kilheeney 
Reviewed-by: Bruce Richardson 
---
 usertools/cpu_layout.py | 13 ++---
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/usertools/cpu_layout.py b/usertools/cpu_layout.py
index 5423c7965f..89a48cec46 100755
--- a/usertools/cpu_layout.py
+++ b/usertools/cpu_layout.py
@@ -1,18 +1,9 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 # Copyright(c) 2017 Cavium, Inc. All rights reserved.
 
-from __future__ import print_function
 import sys
-try:
-xrange # Python 2
-except NameError:
-xrange = range # Python 3
-
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not work 
in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
 
 sockets = []
 cores = []
@@ -21,7 +12,7 @@
 fd = open("{}/kernel_max".format(base_path))
 max_cpus = int(fd.read())
 fd.close()
-for cpu in xrange(max_cpus + 1):
+for cpu in range(max_cpus + 1):
 try:
 fd = open("{}/cpu{}/topology/core_id".format(base_path, cpu))
 except IOError:
-- 
2.17.1

[dpdk-dev] [PATCH v5 2/9] usertools/dpdk-devbind: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Signed-off-by: Louise Kilheeney 
Reviewed-by: Bruce Richardson 
---
 usertools/dpdk-devbind.py | 22 --
 1 file changed, 4 insertions(+), 18 deletions(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 86b6b53c40..bcdc5da881 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -1,19 +1,14 @@
-#! /usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 #
 
-from __future__ import print_function
 import sys
 import os
 import getopt
 import subprocess
 from os.path import exists, abspath, dirname, basename
 
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not work 
in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
-
 # The PCI base class for all devices
 network_class = {'Class': '02', 'Vendor': None, 'Device': None,
 'SVendor': None, 'SDevice': None}
@@ -147,14 +142,6 @@ def usage():
 
 """ % locals())  # replace items from local variables
 
-
-# This is roughly compatible with check_output function in subprocess module
-# which is only available in python 2.7.
-def check_output(args, stderr=None):
-'''Run a command and capture its output'''
-return subprocess.Popen(args, stdout=subprocess.PIPE,
-stderr=stderr).communicate()[0]
-
 # check if a specific kernel module is loaded
 def module_is_loaded(module):
 global loaded_modules
@@ -211,8 +198,7 @@ def get_pci_device_details(dev_id, probe_lspci):
 device = {}
 
 if probe_lspci:
-extra_info = check_output(["lspci", "-vmmks", dev_id]).splitlines()
-
+extra_info = subprocess.check_output(["lspci", "-vmmks", 
dev_id]).splitlines()
 # parse lspci details
 for line in extra_info:
 if len(line) == 0:
@@ -248,7 +234,7 @@ def get_device_details(devices_type):
 # first loop through and read details for all devices
 # request machine readable format, with numeric IDs and String
 dev = {}
-dev_lines = check_output(["lspci", "-Dvmmnnk"]).splitlines()
+dev_lines = subprocess.check_output(["lspci", "-Dvmmnnk"]).splitlines()
 for dev_line in dev_lines:
 if len(dev_line) == 0:
 if device_type_match(dev, devices_type):
@@ -276,7 +262,7 @@ def get_device_details(devices_type):
 # check what is the interface if any for an ssh connection if
 # any to this host, so we can mark it later.
 ssh_if = []
-route = check_output(["ip", "-o", "route"])
+route = subprocess.check_output(["ip", "-o", "route"])
 # filter out all lines for 169.254 routes
 route = "\n".join(filter(lambda ln: not ln.startswith("169.254"),
  route.decode().splitlines()))
-- 
2.17.1

[dpdk-dev] [PATCH v5 5/9] app/test-cmdline: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Cc: Olivier Matz 

Signed-off-by: Louise Kilheeney 

---
v5: fixed python3 issue casuing script to fail
---
 app/test-cmdline/cmdline_test.py  | 9 ++---
 app/test-cmdline/cmdline_test_data.py | 1 +
 mk/rte.sdktest.mk | 2 +-
 3 files changed, 4 insertions(+), 8 deletions(-)

diff --git a/app/test-cmdline/cmdline_test.py b/app/test-cmdline/cmdline_test.py
index 954428e2bf..074fed7e7f 100755
--- a/app/test-cmdline/cmdline_test.py
+++ b/app/test-cmdline/cmdline_test.py
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that runs cmdline_test app and feeds keystrokes into it.
-from __future__ import print_function
 import cmdline_test_data
 import os
 import pexpect
@@ -19,10 +18,6 @@ def runTest(child, test):
 return 0
 child.expect(test["Result"], 1)
 
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not work 
in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
-
 #
 # history test is a special case
 #
@@ -43,7 +38,7 @@ def runHistoryTest(child):
 i = 0
 
 # fill the history with numbers
-while i < history_size / 10:
+while i < int(history_size / 10):
 # add 1 to prevent from parsing as octals
 child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
 # the app will simply print out the number
diff --git a/app/test-cmdline/cmdline_test_data.py 
b/app/test-cmdline/cmdline_test_data.py
index 114d2cb6a0..2d9b3262a6 100644
--- a/app/test-cmdline/cmdline_test_data.py
+++ b/app/test-cmdline/cmdline_test_data.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
diff --git a/mk/rte.sdktest.mk b/mk/rte.sdktest.mk
index 803018ba3a..ad7ef138f0 100644
--- a/mk/rte.sdktest.mk
+++ b/mk/rte.sdktest.mk
@@ -63,7 +63,7 @@ coverage:
@mkdir -p $(AUTOTEST_DIR) ; \
cd $(AUTOTEST_DIR) ; \
if [ -f $(RTE_OUTPUT)/app/test ]; then \
-   python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
+   python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
$(RTE_OUTPUT)/app/cmdline_test; \
ulimit -S -n 100 ; \
python $(RTE_SDK)/app/test/autotest.py \
-- 
2.17.1

[dpdk-dev] [PATCH v5 7/9] devtools: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Cc: Neil Horman 
Cc: Ray Kinsella 

Signed-off-by: Louise Kilheeney 
Acked-by: Ray Kinsella 
---
 devtools/update_version_map_abi.py | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/devtools/update_version_map_abi.py 
b/devtools/update_version_map_abi.py
index 80a61641ed..830e6c58c8 100755
--- a/devtools/update_version_map_abi.py
+++ b/devtools/update_version_map_abi.py
@@ -1,4 +1,4 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2019 Intel Corporation
 
@@ -9,7 +9,6 @@
 from the devtools/update-abi.sh utility.
 """
 
-from __future__ import print_function
 import argparse
 import sys
 import re
@@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
 print("};", file=f_out)
 
 def __main():
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not 
work in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
-
 arg_parser = argparse.ArgumentParser(
 description='Merge versions in linker version script.')
 
-- 
2.17.1

[dpdk-dev] [PATCH v5 8/9] config/arm: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2.

Cc: Thomas Monjalon 

Signed-off-by: Louise Kilheeney 
---
 config/arm/armv8_machine.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/config/arm/armv8_machine.py b/config/arm/armv8_machine.py
index 404866d2f8..1f689d9a83 100755
--- a/config/arm/armv8_machine.py
+++ b/config/arm/armv8_machine.py
@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Cavium, Inc
 
-- 
2.17.1

[dpdk-dev] [PATCH v5 6/9] app/test: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Signed-off-by: Louise Kilheeney 
---
 app/test/autotest.py|  7 +--
 app/test/autotest_data.py   |  1 +
 app/test/autotest_runner.py | 21 -
 app/test/autotest_test_funcs.py |  1 +
 mk/rte.sdktest.mk   |  4 ++--
 5 files changed, 13 insertions(+), 21 deletions(-)

diff --git a/app/test/autotest.py b/app/test/autotest.py
index cf7584ccd7..9eef1efbe5 100644
--- a/app/test/autotest.py
+++ b/app/test/autotest.py
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # Script that uses either test app or qemu controlled by python-pexpect
-from __future__ import print_function
 import autotest_data
 import autotest_runner
 import sys
@@ -17,10 +16,6 @@ def usage():
 usage()
 sys.exit(1)
 
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not work 
in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
-
 target = sys.argv[2]
 
 test_whitelist = None
diff --git a/app/test/autotest_data.py b/app/test/autotest_data.py
index 4b7da45e09..097638941f 100644
--- a/app/test/autotest_data.py
+++ b/app/test/autotest_data.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
diff --git a/app/test/autotest_runner.py b/app/test/autotest_runner.py
index 95e74c760d..998fe57a55 100644
--- a/app/test/autotest_runner.py
+++ b/app/test/autotest_runner.py
@@ -1,10 +1,10 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
 # The main logic behind running autotests in parallel
 
-from __future__ import print_function
-import StringIO
+import io
 import csv
 from multiprocessing import Pool, Queue
 import pexpect
@@ -50,11 +50,7 @@ def first_cpu_on_node(node_nr):
 map(os.path.basename, cpu_path)
 )
 )
-# for compatibility between python 3 and 2 we need to make interable out
-# of filter return as it returns list in python 2 and a generator in 3
-m = next(iter(cpu_name))
-return int(m.group(1))
-
+return int(next(cpu_name).group(1))
 
 pool_child = None  # per-process child
 
@@ -78,7 +74,7 @@ def pool_init(queue, result_queue):
 cmdline = "%s %s" % (cmdline, prefix_cmdline)
 
 # prepare logging of init
-startuplog = StringIO.StringIO()
+startuplog = io.StringIO()
 
 # run test app
 try:
@@ -86,8 +82,7 @@ def pool_init(queue, result_queue):
 print("\n%s %s\n" % ("=" * 20, prefix), file=startuplog)
 print("\ncmdline=%s" % cmdline, file=startuplog)
 
-pool_child = pexpect.spawn(cmdline, logfile=startuplog)
-
+pool_child = pexpect.spawn(cmdline, logfile=startuplog, 
encoding='utf-8')
 # wait for target to boot
 if not wait_prompt(pool_child):
 pool_child.close()
@@ -138,7 +133,7 @@ def run_test(target, test):
 # create log buffer for each test
 # in multiprocessing environment, the logging would be
 # interleaved and will create a mess, hence the buffering
-logfile = StringIO.StringIO()
+logfile = io.StringIO()
 pool_child.logfile = logfile
 
 # make a note when the test started
@@ -210,9 +205,9 @@ def __init__(self, cmdline, target, blacklist, whitelist, 
n_processes):
 # parse the binary for available test commands
 binary = cmdline.split()[0]
 stripped = 'not stripped' not in \
-   subprocess.check_output(['file', binary])
+   subprocess.check_output(['file', binary]).decode()
 if not stripped:
-symbols = subprocess.check_output(['nm', binary]).decode('utf-8')
+symbols = subprocess.check_output(['nm', binary]).decode()
 self.avail_cmds = re.findall('test_register_(\w+)', symbols)
 else:
 self.avail_cmds = None
diff --git a/app/test/autotest_test_funcs.py b/app/test/autotest_test_funcs.py
index 26688b7132..775dfd1dc5 100644
--- a/app/test/autotest_test_funcs.py
+++ b/app/test/autotest_test_funcs.py
@@ -1,3 +1,4 @@
+#!/usr/bin/env python3
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2010-2014 Intel Corporation
 
diff --git a/mk/rte.sdktest.mk b/mk/rte.sdktest.mk
index ad7ef138f0..81eff0c08f 100644
--- a/mk/rte.sdktest.mk
+++ b/mk/rte.sdktest.mk
@@ -49,7 +49,7 @@ test test-fast test-perf test-drivers test-dump:
@mkdir -p $(AUTOTEST_DIR) ; \
cd $(AUTOTEST_DIR) ; \
if [ -f $(RTE_OUTPUT)/app/test ]; then \
-   python $(RTE_SDK)/app/test/autotest.py \
+   python3 $(RTE_SDK)/app/test/autotest.py \
$(RTE_OUTPUT)/app/test \
$(RTE_TARGET) \
$(BLACKL

[dpdk-dev] [PATCH v5 9/9] app/test-bbdev: support python3 only

[Louise Kilheeney]

Changed script to explicitly use python3 only to avoid
maintaining python 2 and removed deprecation notice.

Cc: Nicolas Chautru 

Signed-off-by: Louise Kilheeney 
---
 app/test-bbdev/test-bbdev.py | 7 +--
 1 file changed, 1 insertion(+), 6 deletions(-)

diff --git a/app/test-bbdev/test-bbdev.py b/app/test-bbdev/test-bbdev.py
index 5ae2dc6c49..7d67dbe30a 100755
--- a/app/test-bbdev/test-bbdev.py
+++ b/app/test-bbdev/test-bbdev.py
@@ -1,9 +1,8 @@
-#!/usr/bin/env python
+#!/usr/bin/env python3
 
 # SPDX-License-Identifier: BSD-3-Clause
 # Copyright(c) 2017 Intel Corporation
 
-from __future__ import print_function
 import sys
 import os
 import argparse
@@ -16,10 +15,6 @@ def kill(process):
 print("ERROR: Test app timed out")
 process.kill()
 
-if sys.version_info.major < 3:
-print("WARNING: Python 2 is deprecated for use in DPDK, and will not work 
in future releases.", file=sys.stderr)
-print("Please use Python 3 instead", file=sys.stderr)
-
 if "RTE_SDK" in os.environ:
 dpdk_path = os.environ["RTE_SDK"]
 else:
-- 
2.17.1

Re: [dpdk-dev] [PATCH v2] usertools/dpdk-devbind: add support for PCI wildcards

[Burakov, Anatoly]

On 20-Aug-20 4:43 PM, Bruce Richardson wrote:

When binding or unbinding a range of devices, it can be useful to use
wildcards to specify the devices rather than repeating the same prefix
multiple times. We can use the python "glob" module to give us this
functionality - at least for PCI devices - by checking /sys for matching
files.

Examples of use from my system:

 ./dpdk-devbind.py -b vfio-pci 80:04.*
 ./dpdk-devbind.py -u 80:04.[2-7]

The first example binds eight devices, 80:04.0..80:04.7, to vfio-pci. The
second then unbinds six of those devices, 80:04.2..80:04.7, from any
driver.

Signed-off-by: Bruce Richardson 
Tested-by: Ferruh Yigit 
---
V2: added help text additions
---
  usertools/dpdk-devbind.py | 16 
  1 file changed, 16 insertions(+)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 86b6b53c40..d13defbe1a 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -8,6 +8,7 @@
  import os
  import getopt
  import subprocess
+from glob import glob
  from os.path import exists, abspath, dirname, basename
  
  if sys.version_info.major < 3:

@@ -89,6 +90,8 @@ def usage():
  where DEVICE1, DEVICE2 etc, are specified via PCI "domain:bus:slot.func" 
syntax
  or "bus:slot.func" syntax. For devices bound to Linux kernel drivers, they may
  also be referred to by Linux interface name e.g. eth0, eth1, em0, em1, etc.
+If devices are specified using PCI bus:device:func format, then
+shell wildcards and ranges may be used, e.g. 80:04.*, 80:04.[0-3]
  
  Options:

  --help, --usage:
@@ -145,6 +148,9 @@ def usage():
  To bind :02:00.0 and :02:00.1 to the ixgbe kernel driver
  %(argv0)s -b ixgbe 02:00.0 02:00.1
  
+To bind all funcions on device :02:00 to ixgbe kernel driver

+%(argv0)s -b ixgbe 02:00.*
+
  """ % locals())  # replace items from local variables
  
  
@@ -689,6 +695,16 @@ def parse_args():

  else:
  b_flag = arg
  
+# resolve any PCI globs in the args

+new_args = []
+sysfs_path = "/sys/bus/pci/devices/"
+for arg in args:
+globbed_arg = glob(sysfs_path + arg) + glob(sysfs_path + ":" + arg)


Also, could be

glob_path = arg if arg.startswith(":") else ":" + arg
globbed_arg = glob(os.path.join(sysfs_path, glob_path))

No need to glob twice :)


+if globbed_arg:
+new_args.extend([a[len(sysfs_path):] for a in globbed_arg])
+else:
+new_args.append(arg)
+args = new_args
  
  def do_arg_actions():

  '''do the actual action requested by the user'''




--
Thanks,
Anatoly

Re: [dpdk-dev] [PATCH v5 5/9] app/test-cmdline: support python3 only

[Bruce Richardson]

On Fri, Aug 21, 2020 at 10:14:48AM +0100, Louise Kilheeney wrote:
> Changed script to explicitly use python3 only to avoid
> maintaining python 2 and removed deprecation notice.
> 
> Cc: Olivier Matz 
> 
> Signed-off-by: Louise Kilheeney 
> 
> ---
> v5: fixed python3 issue casuing script to fail

It would be good to add a little more detail of the failure and fix to help
reviewers, I believe the fix is the typecast to int() around the division
to avoid floating point numbers messing up the loop count.

Reviewed-by: Bruce Richardson 

> ---
>  app/test-cmdline/cmdline_test.py  | 9 ++---
>  app/test-cmdline/cmdline_test_data.py | 1 +
>  mk/rte.sdktest.mk | 2 +-
>  3 files changed, 4 insertions(+), 8 deletions(-)
> 
> diff --git a/app/test-cmdline/cmdline_test.py 
> b/app/test-cmdline/cmdline_test.py
> index 954428e2bf..074fed7e7f 100755
> --- a/app/test-cmdline/cmdline_test.py
> +++ b/app/test-cmdline/cmdline_test.py
> @@ -1,9 +1,8 @@
> -#!/usr/bin/env python
> +#!/usr/bin/env python3
>  # SPDX-License-Identifier: BSD-3-Clause
>  # Copyright(c) 2010-2014 Intel Corporation
>  
>  # Script that runs cmdline_test app and feeds keystrokes into it.
> -from __future__ import print_function
>  import cmdline_test_data
>  import os
>  import pexpect
> @@ -19,10 +18,6 @@ def runTest(child, test):
>  return 0
>  child.expect(test["Result"], 1)
>  
> -if sys.version_info.major < 3:
> -print("WARNING: Python 2 is deprecated for use in DPDK, and will not 
> work in future releases.", file=sys.stderr)
> -print("Please use Python 3 instead", file=sys.stderr)
> -
>  #
>  # history test is a special case
>  #
> @@ -43,7 +38,7 @@ def runHistoryTest(child):
>  i = 0
>  
>  # fill the history with numbers
> -while i < history_size / 10:
> +while i < int(history_size / 10):
>  # add 1 to prevent from parsing as octals
>  child.send("1" + str(i).zfill(8) + cmdline_test_data.ENTER)
>  # the app will simply print out the number
> diff --git a/app/test-cmdline/cmdline_test_data.py 
> b/app/test-cmdline/cmdline_test_data.py
> index 114d2cb6a0..2d9b3262a6 100644
> --- a/app/test-cmdline/cmdline_test_data.py
> +++ b/app/test-cmdline/cmdline_test_data.py
> @@ -1,3 +1,4 @@
> +#!/usr/bin/env python3
>  # SPDX-License-Identifier: BSD-3-Clause
>  # Copyright(c) 2010-2014 Intel Corporation
>  
> diff --git a/mk/rte.sdktest.mk b/mk/rte.sdktest.mk
> index 803018ba3a..ad7ef138f0 100644
> --- a/mk/rte.sdktest.mk
> +++ b/mk/rte.sdktest.mk
> @@ -63,7 +63,7 @@ coverage:
>   @mkdir -p $(AUTOTEST_DIR) ; \
>   cd $(AUTOTEST_DIR) ; \
>   if [ -f $(RTE_OUTPUT)/app/test ]; then \
> - python $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
> + python3 $(RTE_SDK)/test/cmdline_test/cmdline_test.py \
>   $(RTE_OUTPUT)/app/cmdline_test; \
>   ulimit -S -n 100 ; \
>   python $(RTE_SDK)/app/test/autotest.py \
> -- 
> 2.17.1
> 

Re: [dpdk-dev] [PATCH v2 20/37] doc: remove references to make in bbdev guides

[Bruce Richardson]

On Fri, Aug 21, 2020 at 02:22:14AM +, Chautru, Nicolas wrote:
> From: Power, Ciara 
> > Make is no longer supported for compiling DPDK, references are now removed
> > in the documentation.
> > 
> > Reviewed-by: Kevin Laatz 
> > Signed-off-by: Ciara Power 
> > ---
> >  doc/guides/bbdevs/fpga_5gnr_fec.rst |  7 ++-
> > doc/guides/bbdevs/fpga_lte_fec.rst  |  7 ++-
> >  doc/guides/bbdevs/turbo_sw.rst  | 15 ---
> >  3 files changed, 4 insertions(+), 25 deletions(-)
> > 
> > diff --git a/doc/guides/bbdevs/fpga_5gnr_fec.rst
> > b/doc/guides/bbdevs/fpga_5gnr_fec.rst
> > index 6760391e8c..8e00c4ef22 100644
> > --- a/doc/guides/bbdevs/fpga_5gnr_fec.rst
> > +++ b/doc/guides/bbdevs/fpga_5gnr_fec.rst
> > @@ -51,10 +51,7 @@ FPGA 5GNR FEC does not support the following:
> >  Installation
> >  
> > 
> > -Section 3 of the DPDK manual provides instructions on installing and 
> > compiling
> > DPDK. The -default set of bbdev compile flags may be found in
> > config/common_base, where for example -the flag to build the FPGA 5GNR FEC
> > device, ``CONFIG_RTE_LIBRTE_PMD_BBDEV_FPGA_5GNR_FEC``,
> > -is already set.
> > +Section 3 of the DPDK manual provides instructions on installing and 
> > compiling
> > DPDK.
> > 
> >  DPDK requires hugepages to be configured as detailed in section 2 of the 
> > DPDK
> > manual.
> >  The bbdev test application has been tested with a configuration 40 x 1GB
> > hugepages. The @@ -94,7 +91,7 @@ the UIO driver by repeating this
> > command for every function.
> >  .. code-block:: console
> > 
> >cd 
> > -  insmod ./build/kmod/igb_uio.ko
> > +  insmod .//kernel/linux/igb_uio/igb_uio.ko
> >echo "8086 0d8f" > /sys/bus/pci/drivers/igb_uio/new_id
> >lspci -vd8086:0d8f
> > 
> > diff --git a/doc/guides/bbdevs/fpga_lte_fec.rst
> > b/doc/guides/bbdevs/fpga_lte_fec.rst
> > index fdc8a76981..14ffa0ee14 100644
> > --- a/doc/guides/bbdevs/fpga_lte_fec.rst
> > +++ b/doc/guides/bbdevs/fpga_lte_fec.rst
> > @@ -50,10 +50,7 @@ FPGA LTE FEC does not support the following:
> >  Installation
> >  --
> > 
> > -Section 3 of the DPDK manual provides instructions on installing and 
> > compiling
> > DPDK. The -default set of bbdev compile flags may be found in
> > config/common_base, where for example -the flag to build the FPGA LTE FEC
> > device, ``CONFIG_RTE_LIBRTE_PMD_BBDEV_FPGA_LTE_FEC``, is already -set.
> > +Section 3 of the DPDK manual provides instructions on installing and 
> > compiling
> > DPDK.
> > 
> >  DPDK requires hugepages to be configured as detailed in section 2 of the 
> > DPDK
> > manual.
> >  The bbdev test application has been tested with a configuration 40 x 1GB
> > hugepages. The @@ -93,7 +90,7 @@ the UIO driver by repeating this
> > command for every function.
> >  .. code-block:: console
> > 
> >cd 
> > -  insmod ./build/kmod/igb_uio.ko
> > +  insmod .//kernel/linux/igb_uio/igb_uio.ko
> >echo "1172 5052" > /sys/bus/pci/drivers/igb_uio/new_id
> >lspci -vd1172:
> > 
> > diff --git a/doc/guides/bbdevs/turbo_sw.rst b/doc/guides/bbdevs/turbo_sw.rst
> > index 20620c2e20..3b93157890 100644
> > --- a/doc/guides/bbdevs/turbo_sw.rst
> > +++ b/doc/guides/bbdevs/turbo_sw.rst
> > @@ -12,14 +12,6 @@ Note that the driver can also be built without any
> > dependency with reduced  functionality for maintenance purpose.
> > 
> >  To enable linking to the SDK libraries see detailed installation section 
> > below.
> > -Two flags can be enabled depending on whether the target machine can
> > support
> > -AVX2 and AVX512 instructions sets and the related SDK libraries for 
> > vectorized
> > -signal processing functions are installed :
> > -- CONFIG_RTE_BBDEV_SDK_AVX2
> > -- CONFIG_RTE_BBDEV_SDK_AVX512
> > -By default these 2 flags are disabled by default. For AVX2 machine and SDK 
> > -
> > library installed then the first flag can be enabled. For AVX512 machine 
> > and -
> > SDK library installed then both flags can be enabled for full real time 
> > capability.
> 
> Hi, 
> We are losing information here. Even if the configuration options should not 
> be documented based on an explicit reference make and config_base assumptions 
> (agreed that they are no longer there), the steps should still capture the 
> conf to be passed to meson so that the PMD can still be built with the 
> avx2/avx512 intel libraries (ie. not the case by default due to additional 
> dependency to retrieve the related libraries). 
> Thanks
> Nic

Hi Nic,

Can you perhaps suggest the appropriate wording to use? Given the size of
the patchset, I think we need individual maintainers to help provide the
appropriate doc adjustments.

Thanks,
/Bruce

Re: [dpdk-dev] [PATCH] net/ark: fix meson build

[Ferruh Yigit]

On 8/20/2020 4:41 PM, Ed Czeck wrote:
> On Thu, Aug 20, 2020 at 7:16 AM Ferruh Yigit  wrote:
>>
> ...
>>
>> Logging can be controlled in runtime, that is what we should use.
>> In data path, we use compile time flags because of the performance issues. 
>> So OK
>> to have 'CONFIG_RTE_LIBRTE_ARK_DEBUG_RX' & 'CONFIG_RTE_LIBRTE_ARK_DEBUG_TX' 
>> as
>> compile time flag, but why having compile time flag for rest?
> 
> Agreed.  We'll remove most of these macro in the next commit pushing
> the behavior
> into run-time log control.
> 
>>>
>>> +Note that enabling debugging options may affect system performance.
>>> +These options may be set by specifying them in CFLAG
>>> +environment before the meson build set.   E.g.::
>>> +
>>> +export CFLAGS="-DARK_DEBUG_TRACE"
>>> +meson build
>>> +
>>
>> When you passed the flag as above, it is still global to all components in 
>> the
>> DPDK, this is not just for ark. What is the motivation to remove the
>> "RET_LIBRTE_" prefix?
> 
> There are 2 issues here.
> 1) With the makefile flow, users could add other configurations with
> the documented
> and recommended sed commands.  I.e.,
> sed -ri 's/(CONFIG_RTE_LIBRTE_ARK_DEBUG_TRACE)=n/\1=y/' build/.config
> The makefiles took care of everything from there.  This no longer works with 
> the
> meson build flow.  The solution is to set that macro passing it in the
> CFLAG environment.
> We're open to other recommendations.

Yes compile time flags are harder to use with meson build system, and this is
kind of intentional because of above mentioned reasons.

It is possible to add compile time option as meson config option if we must, but
that is not suggested, the intention is get rid of compile time flags as much as
we can, and we were already very picky to accept new compile time flags to make
build for some time.

> 2) Is there an advantage of promoting a PMD macro to a global macro?
> It seems to add to the noise of dpdk configuration and there are many
> PMD specific macros throughout the code base.

If CFLAG used to pass macros to multiple components, or if there is a bigger
build environment that builds both application and DPDK, CFLAGS is common flag
and I think better to keep the DPDK namespace for DPDK macros. At least let me
say the way that I don't see the motivation to remove the DPDK namespace from
macros.

> 
>>
>>>
>>>  Building DPDK
>>>  -
>>> diff --git a/drivers/net/ark/ark_logs.h b/drivers/net/ark/ark_logs.h
>>> index 44aac6102..125583475 100644
>>> --- a/drivers/net/ark/ark_logs.h
>>> +++ b/drivers/net/ark/ark_logs.h
>>> @@ -6,14 +6,12 @@
>>>  #define _ARK_DEBUG_H_
>>>
>>>  #include 
>>> -#include 
>>> -
>>>
>>>  /* Configuration option to pad TX packets to 60 bytes */
>>> -#ifdef RTE_LIBRTE_ARK_PAD_TX
>>> -#define ARK_TX_PAD_TO_60   1
>>> -#else
>>> +#ifdef ARK_NOPAD_TX
>>>  #define ARK_TX_PAD_TO_60   0
>>> +#else
>>> +#define ARK_TX_PAD_TO_60   1
>>>  #endif
>>
>> So you don't want to convert this to runtime configuration.
>>
>> The point we are reducing compile time flags:
>> 1) It forks the code paths and by time it leave not tested, even not compiled
>> code paths which may cause rotten code by time.
>>
>> 2) Multiple code paths will lead deployment problems. When you deploy an
>> application, you won't able to change the compile time configuration in 
>> customer
>> environment and need to re-compile (most probably re-test) and re-deploy it.
>> Also there is not easy way to figure out from binary in customer environment
>> that with which compile time flags it has been built.
>>
>> Switching to CFLAGS="..." doesn't make above concerns go away and indeed it
>> makes (1) worst since hides the config options within the driver. Previously 
>> it
>> was possible to trigger each config option and do testing using scripts, now
>> since config options are hidden in driver we can't do even that.
>>
>> Can you please detail why "ARK_TX_PAD_TO_60" is needed exactly?
>> And can you please justify why it has to be compile time config option?
>>
> The need to pad packets is dependent on the underlying FPGA hardware
> implementation which lies outside the control of our deliverables.  Thus we
> leave control up to our customer and how they deliver and deploy DPDK.  This
> needs to be a compile-time macro since the code executes within the
> per-packet processing under rte_eth_tx_burst().

It is still possible to use runtime config by having two burst functions, one
with padding and without padding support, which is selected based on provided
runtime flag. Runtime parameter can provide the padding size.
But agree that adds more complexity.

> 
> We can change the macro to ARK_MIN_TX_PKTLEN, which should have zero
> overhead when set to 0.  (I'm assuming that compiles will remove if
> (unsigned < 0) blocks.)  Should this be an RTE_LIBRTE macro?  How does
> a user change this during compile?
> 
>> <...>
>>
>>> @@ -11,3 +11,5 @@ sources = files('ark_ddm.c',
>>>   'ark_pktgen.c',
>>>   'ark_rqp.

Re: [dpdk-dev] [PATCH 2/2] net/ark remove ARK_TX_PAD_TO_60 configuration macro

[Ferruh Yigit]

On 8/20/2020 10:55 PM, Ed Czeck wrote:
> Update documenation as needed.
> 
> Signed-off-by: Ed Czeck 
> ---
>  doc/guides/nics/ark.rst |  3 ---
>  drivers/net/ark/ark_ethdev_tx.c | 42 ++---
>  drivers/net/ark/ark_logs.h  |  8 ---
>  3 files changed, 23 insertions(+), 30 deletions(-)
> 
> diff --git a/doc/guides/nics/ark.rst b/doc/guides/nics/ark.rst
> index c3ffcbbc2..0f96ebd2c 100644
> --- a/doc/guides/nics/ark.rst
> +++ b/doc/guides/nics/ark.rst
> @@ -129,9 +129,6 @@ Configuration Information
> * **CONFIG_RTE_LIBRTE_ARK_PMD** (default y): Enables or disables inclusion
>   of the ARK PMD driver in the DPDK compilation.
>  
> -   * **CONFIG_RTE_LIBRTE_ARK_PAD_TX** (default y):  When enabled TX
> - packets are padded to 60 bytes to support downstream MACS.
> -

It may be useful to document what replaced it and how to use it:

export CFLAGS="-DARK_MIN_TX_PKTLEN=60"
meson build

And as mention in reply to prev version, I think better to keep 'RTE_LIBRTE_',
as 'RTE_LIBRTE_ARK_MIN_TX_PKTLEN'

Re: [dpdk-dev] [PATCH v2 20/37] doc: remove references to make in bbdev guides

[Power, Ciara]

Hi Nicolas,


>-Original Message-
>From: Chautru, Nicolas 
>Sent: Friday 21 August 2020 03:13
>To: Power, Ciara ; dev@dpdk.org
>Cc: Mcnamara, John ; Kovacevic, Marko
>
>Subject: RE: [PATCH v2 20/37] doc: remove references to make in bbdev
>guides
>
>> From: Power, Ciara  Make is no longer supported
>> for compiling DPDK, references are now removed in the documentation.
>>
>> Reviewed-by: Kevin Laatz 
>> Signed-off-by: Ciara Power 
>> ---
>>  doc/guides/bbdevs/fpga_5gnr_fec.rst |  7 ++-
>> doc/guides/bbdevs/fpga_lte_fec.rst  |  7 ++-
>>  doc/guides/bbdevs/turbo_sw.rst  | 15 ---
>>  3 files changed, 4 insertions(+), 25 deletions(-)
>>

>>cd 
>> -  insmod ./build/kmod/igb_uio.ko
>> +  insmod .//kernel/linux/igb_uio/igb_uio.ko
>
>Hi,
>Should this more specific? What do you mean by  here? Is this
>defined somewhere else explicitly in doc?
>In other commit (with the python script) the assumption is that meson will
>install under /build directory with hard coded path,
>ie. not forcing to other location.
>Can you clarify so that we are comprehensive and avoid confusion?
>Thanks
>Nic
>
>
>

The "" here represents the meson build folder, which is named by the 
user so can vary.
In the python script, I added "build" as the hardcoded path for the build 
folder, based on the example in the prog_guide/build-sdk-meson guide, which 
chooses to name the folder "build".

Thanks,
Ciara

Re: [dpdk-dev] [EXT] Re: [PATCH v5 1/2] ethdev: add level support for RSS offload types

[Kiran Kumar Kokkilagadda]

From: Ajit Khaparde 
Sent: Thursday, August 20, 2020 8:49 AM
To: Kiran Kumar Kokkilagadda 
Cc: Wenzhuo Lu ; Beilei Xing ; 
Bernard Iremonger ; Thomas Monjalon 
; Ferruh Yigit ; Andrew Rybchenko 
; dpdk-dev ; Jerin Jacob Kollanukkaran 
; Ori Kam ; Ziyang Xuan 
; Xiaoyun Wang ; Guoyang 
Zhou ; Rosen Xu ; 
jia@intel.com; Rasesh Mody ; Shahed Shaikh 
; Nithin Kumar Dabilpuram ; 
Qiming Yang ; Qi Zhang ; Wiles, 
Keith ; Hemant Agrawal ; Sachin 
Saxena ; Zhao1, Wei ; John Daley 
; Hyong Youb Kim ; Chas Williams 
; Matan Azrad ; Shahaf Shuler 
; Viacheslav Ovsiienko ; Rahul 
Lakkireddy ; Gaetan Rivet ; Liron 
Himi ; Jingjing Wu ; Wei Hu (Xavier 
; humi...@huawei.com; yisen.zhu...@huawei.com; Somnath 
Kotur ; Singh, Jasvinder 
; Dumitrescu, Cristian 

Subject: [EXT] Re: [dpdk-dev][PATCH v5 1/2] ethdev: add level support for RSS 
offload types

External Email



On Tue, Aug 18, 2020 at 11:05 PM 
mailto:kirankum...@marvell.com>> wrote:
From: Kiran Kumar K mailto:kirankum...@marvell.com>>

This patch reserves 2 bits as input selection to select Inner and
outer layers for RSS computation. It is combined with existing
ETH_RSS_* to choose Inner or outer layers for L2, L3 and L4.
This functionality already exists in rte_flow through level parameter in
RSS action configuration rte_flow_action_rss.

Signed-off-by: Kiran Kumar K 
mailto:kirankum...@marvell.com>>
---
V5 Changes:
* Added support to set rss level type from port config in testpmd

 app/test-pmd/cmdline.c | 11 ++-
 app/test-pmd/parameters.c  |  6 ++
Can you split this into testpmd and ethdev patches.
Becomes easy to reference, fix.

Will send V6.

 lib/librte_ethdev/rte_ethdev.h | 27 +++
 3 files changed, 43 insertions(+), 1 deletion(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 0a6ed85f3..4eafee8c8 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -2334,7 +2334,16 @@ cmd_config_rss_parsed(void *parsed_result,
rss_conf.rss_hf = ETH_RSS_GTPU;
else if (!strcmp(res->value, "none"))
rss_conf.rss_hf = 0;
-   else if (!strcmp(res->value, "default"))
+   else if (!strcmp(res->value, "level-inner")) {
+   rss_hf &= (~ETH_RSS_LEVEL_MASK);
+   rss_conf.rss_hf = (rss_hf | ETH_RSS_LEVEL_INNER);
+   } else if (!strcmp(res->value, "level-outer")) {
+   rss_hf &= (~ETH_RSS_LEVEL_MASK);
+   rss_conf.rss_hf = (rss_hf | ETH_RSS_LEVEL_OUTER);
+   } else if (!strcmp(res->value, "level-inner-outer")) {
+   rss_hf &= (~ETH_RSS_LEVEL_MASK);
+   rss_conf.rss_hf = (rss_hf | ETH_RSS_LEVEL_INNER_OUTER);
+   } else if (!strcmp(res->value, "default"))
use_default = 1;
else if (isdigit(res->value[0]) && atoi(res->value) > 0 &&
atoi(res->value) < 64)
diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index 7cb0e3d6e..5f669ff24 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -632,6 +632,8 @@ launch_args_parse(int argc, char** argv)
{ "forward-mode",   1, 0, 0 },
{ "rss-ip", 0, 0, 0 },
{ "rss-udp",0, 0, 0 },
+   { "rss-outer",  0, 0, 0 },
+   { "rss-inner-outer",0, 0, 0 },
{ "rxq",1, 0, 0 },
{ "txq",1, 0, 0 },
{ "rxd",1, 0, 0 },
@@ -1051,6 +1053,10 @@ launch_args_parse(int argc, char** argv)
rss_hf = ETH_RSS_IP;
if (!strcmp(lgopts[opt_idx].name, "rss-udp"))
rss_hf = ETH_RSS_UDP;
+   if (!strcmp(lgopts[opt_idx].name, "rss-outer"))
+   rss_hf |= ETH_RSS_LEVEL_OUTER;
+   if (!strcmp(lgopts[opt_idx].name, "rss-inner-outer"))
+   rss_hf |= ETH_RSS_LEVEL_INNER_OUTER;
if (!strcmp(lgopts[opt_idx].name, "rxq")) {
n = atoi(optarg);
if (n >= 0 && check_nb_rxq((queueid_t)n) == 0)
diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index d29930fd8..28184cc85 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -552,6 +552,33 @@ struct rte_eth_rss_conf {
 #define RTE_ETH_RSS_L3_PRE64  (1ULL << 53)
 #define RTE_ETH_RSS_L3_PRE96  (1ULL << 52)

+/*
+ * We use the following macros to combine with the above layers to choose
+ * inner and outer layers or both for RSS computation.
+ * Note: Default is 0: inner layers, 1: outer layers, 2: both
+ * bit 50 and 51 are reserved for this.
+ */
+
+/**
+ * Level 0, It basically stands for the innermost encaps

[dpdk-dev] [PATCH v6 2/3] app/testpmd: support ethdev rss level config

[kirankumark]

From: Kiran Kumar K 

Adding support to set RSS level from ethdev config.
level-inner is default and will set the RSS level to inner layers.
level-outer will set the RSS level to outer layers.
level-inner-outer will set the RSS level to both inner and outer
layers.

Signed-off-by: Kiran Kumar K 
---
 app/test-pmd/cmdline.c| 11 ++-
 app/test-pmd/parameters.c |  6 ++
 2 files changed, 16 insertions(+), 1 deletion(-)

diff --git a/app/test-pmd/cmdline.c b/app/test-pmd/cmdline.c
index 0a6ed85f3..4eafee8c8 100644
--- a/app/test-pmd/cmdline.c
+++ b/app/test-pmd/cmdline.c
@@ -2334,7 +2334,16 @@ cmd_config_rss_parsed(void *parsed_result,
rss_conf.rss_hf = ETH_RSS_GTPU;
else if (!strcmp(res->value, "none"))
rss_conf.rss_hf = 0;
-   else if (!strcmp(res->value, "default"))
+   else if (!strcmp(res->value, "level-inner")) {
+   rss_hf &= (~ETH_RSS_LEVEL_MASK);
+   rss_conf.rss_hf = (rss_hf | ETH_RSS_LEVEL_INNER);
+   } else if (!strcmp(res->value, "level-outer")) {
+   rss_hf &= (~ETH_RSS_LEVEL_MASK);
+   rss_conf.rss_hf = (rss_hf | ETH_RSS_LEVEL_OUTER);
+   } else if (!strcmp(res->value, "level-inner-outer")) {
+   rss_hf &= (~ETH_RSS_LEVEL_MASK);
+   rss_conf.rss_hf = (rss_hf | ETH_RSS_LEVEL_INNER_OUTER);
+   } else if (!strcmp(res->value, "default"))
use_default = 1;
else if (isdigit(res->value[0]) && atoi(res->value) > 0 &&
atoi(res->value) < 64)
diff --git a/app/test-pmd/parameters.c b/app/test-pmd/parameters.c
index 7cb0e3d6e..5f669ff24 100644
--- a/app/test-pmd/parameters.c
+++ b/app/test-pmd/parameters.c
@@ -632,6 +632,8 @@ launch_args_parse(int argc, char** argv)
{ "forward-mode",   1, 0, 0 },
{ "rss-ip", 0, 0, 0 },
{ "rss-udp",0, 0, 0 },
+   { "rss-outer",  0, 0, 0 },
+   { "rss-inner-outer",0, 0, 0 },
{ "rxq",1, 0, 0 },
{ "txq",1, 0, 0 },
{ "rxd",1, 0, 0 },
@@ -1051,6 +1053,10 @@ launch_args_parse(int argc, char** argv)
rss_hf = ETH_RSS_IP;
if (!strcmp(lgopts[opt_idx].name, "rss-udp"))
rss_hf = ETH_RSS_UDP;
+   if (!strcmp(lgopts[opt_idx].name, "rss-outer"))
+   rss_hf |= ETH_RSS_LEVEL_OUTER;
+   if (!strcmp(lgopts[opt_idx].name, "rss-inner-outer"))
+   rss_hf |= ETH_RSS_LEVEL_INNER_OUTER;
if (!strcmp(lgopts[opt_idx].name, "rxq")) {
n = atoi(optarg);
if (n >= 0 && check_nb_rxq((queueid_t)n) == 0)
-- 
2.25.1

[dpdk-dev] [PATCH v6 3/3] net/octeontx2: add rss hash level support

[kirankumark]

From: Kiran Kumar K 

Add support to choose rss hash level from ethdev rss config.

Signed-off-by: Kiran Kumar K 
---
 drivers/net/octeontx2/otx2_ethdev.h | 2 +-
 drivers/net/octeontx2/otx2_rss.c| 9 +++--
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/drivers/net/octeontx2/otx2_ethdev.h 
b/drivers/net/octeontx2/otx2_ethdev.h
index e9efe52bb..a11411239 100644
--- a/drivers/net/octeontx2/otx2_ethdev.h
+++ b/drivers/net/octeontx2/otx2_ethdev.h
@@ -119,7 +119,7 @@
 #define NIX_RSS_OFFLOAD(ETH_RSS_PORT | ETH_RSS_IP | 
ETH_RSS_UDP |\
 ETH_RSS_TCP | ETH_RSS_SCTP | \
 ETH_RSS_TUNNEL | ETH_RSS_L2_PAYLOAD | \
-NIX_RSS_L3_L4_SRC_DST)
+NIX_RSS_L3_L4_SRC_DST | ETH_RSS_LEVEL_MASK)
 
 #define NIX_TX_OFFLOAD_CAPA ( \
DEV_TX_OFFLOAD_MBUF_FAST_FREE   | \
diff --git a/drivers/net/octeontx2/otx2_rss.c b/drivers/net/octeontx2/otx2_rss.c
index d859937e6..194f4e8af 100644
--- a/drivers/net/octeontx2/otx2_rss.c
+++ b/drivers/net/octeontx2/otx2_rss.c
@@ -323,6 +323,7 @@ otx2_nix_rss_hash_update(struct rte_eth_dev *eth_dev,
 struct rte_eth_rss_conf *rss_conf)
 {
struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev);
+   uint8_t rss_hash_level;
uint32_t flowkey_cfg;
uint8_t alg_idx;
int rc;
@@ -339,7 +340,9 @@ otx2_nix_rss_hash_update(struct rte_eth_dev *eth_dev,
otx2_nix_rss_set_key(dev, rss_conf->rss_key,
 (uint32_t)rss_conf->rss_key_len);
 
-   flowkey_cfg = otx2_rss_ethdev_to_nix(dev, rss_conf->rss_hf, 0);
+   rss_hash_level = ETH_RSS_LEVEL(rss_conf->rss_hf);
+   flowkey_cfg =
+   otx2_rss_ethdev_to_nix(dev, rss_conf->rss_hf, rss_hash_level);
 
rc = otx2_rss_set_hf(dev, flowkey_cfg, &alg_idx,
 NIX_DEFAULT_RSS_CTX_GROUP,
@@ -375,6 +378,7 @@ otx2_nix_rss_config(struct rte_eth_dev *eth_dev)
 {
struct otx2_eth_dev *dev = otx2_eth_pmd_priv(eth_dev);
uint32_t idx, qcnt = eth_dev->data->nb_rx_queues;
+   uint8_t rss_hash_level;
uint32_t flowkey_cfg;
uint64_t rss_hf;
uint8_t alg_idx;
@@ -399,7 +403,8 @@ otx2_nix_rss_config(struct rte_eth_dev *eth_dev)
}
 
rss_hf = eth_dev->data->dev_conf.rx_adv_conf.rss_conf.rss_hf;
-   flowkey_cfg = otx2_rss_ethdev_to_nix(dev, rss_hf, 0);
+   rss_hash_level = ETH_RSS_LEVEL(rss_hf);
+   flowkey_cfg = otx2_rss_ethdev_to_nix(dev, rss_hf, rss_hash_level);
 
rc = otx2_rss_set_hf(dev, flowkey_cfg, &alg_idx,
 NIX_DEFAULT_RSS_CTX_GROUP,
-- 
2.25.1

[dpdk-dev] [PATCH v6 1/3] ethdev: add level support for RSS offload types

[kirankumark]

From: Kiran Kumar K 

This patch reserves 2 bits as input selection to select Inner and
outer layers for RSS computation. It is combined with existing
ETH_RSS_* to choose Inner or outer layers for L2, L3 and L4.
This functionality already exists in rte_flow through level parameter in
RSS action configuration rte_flow_action_rss.

Signed-off-by: Kiran Kumar K 
---
V7 Changes:
* Split ethdev and testpmd changes into seperate patches.

 lib/librte_ethdev/rte_ethdev.h | 27 +++
 1 file changed, 27 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.h b/lib/librte_ethdev/rte_ethdev.h
index 70295d7ab..2a3a76d37 100644
--- a/lib/librte_ethdev/rte_ethdev.h
+++ b/lib/librte_ethdev/rte_ethdev.h
@@ -552,6 +552,33 @@ struct rte_eth_rss_conf {
 #define RTE_ETH_RSS_L3_PRE64  (1ULL << 53)
 #define RTE_ETH_RSS_L3_PRE96  (1ULL << 52)

+/*
+ * We use the following macros to combine with the above layers to choose
+ * inner and outer layers or both for RSS computation.
+ * Note: Default is 0: inner layers, 1: outer layers, 2: both
+ * bit 50 and 51 are reserved for this.
+ */
+
+/**
+ * Level 0, It basically stands for the innermost encapsulation level RSS
+ * can be performed on according to PMD and device capabilities.
+ */
+#define ETH_RSS_LEVEL_INNER(0ULL << 50)
+/**
+ * Level 1, It basically stands for the outermost encapsulation level RSS
+ * can be performed on according to PMD and device capabilities.
+ */
+#define ETH_RSS_LEVEL_OUTER(1ULL << 50)
+/**
+ * Level 2, It basically stands for the both inner and outermost
+ * encapsulation level RSS can be performed on according to PMD and
+ * device capabilities.
+ */
+#define ETH_RSS_LEVEL_INNER_OUTER  (2ULL << 50)
+#define ETH_RSS_LEVEL_MASK(3ULL << 50)
+
+#define ETH_RSS_LEVEL(rss_hf) ((rss_hf & ETH_RSS_LEVEL_MASK) >> 50)
+
 /**
  * For input set change of hash filter, if SRC_ONLY and DST_ONLY of
  * the same level are used simultaneously, it is the same case as
--
2.25.1

[dpdk-dev] [PATCH] net: calculate checksums for packets with IPv4 options

[Michael Pfeiffer]

Currently, rte_ipv4_cksum() and rte_ipv4_udptcp_cksum() assume all IPv4
headers have sizeof(struct rte_ipv4_hdr) bytes. This is not true for
those (rare) packets with IPv4 options. Thus, both IPv4 and TCP/UDP
checksums are calculated wrong.

This patch fixes the issue by using the actual IPv4 header length from
the packet's IHL field.

Signed-off-by: Michael Pfeiffer 
---
 lib/librte_net/rte_ip.h | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/librte_net/rte_ip.h b/lib/librte_net/rte_ip.h
index fcd1eb342..6c3ff2603 100644
--- a/lib/librte_net/rte_ip.h
+++ b/lib/librte_net/rte_ip.h
@@ -269,7 +269,7 @@ static inline uint16_t
 rte_ipv4_cksum(const struct rte_ipv4_hdr *ipv4_hdr)
 {
uint16_t cksum;
-   cksum = rte_raw_cksum(ipv4_hdr, sizeof(struct rte_ipv4_hdr));
+   cksum = rte_raw_cksum(ipv4_hdr, (ipv4_hdr->version_ihl & 0xf) * 4);
return (uint16_t)~cksum;
 }
 
@@ -311,7 +311,7 @@ rte_ipv4_phdr_cksum(const struct rte_ipv4_hdr *ipv4_hdr, 
uint64_t ol_flags)
} else {
psd_hdr.len = rte_cpu_to_be_16(
(uint16_t)(rte_be_to_cpu_16(ipv4_hdr->total_length)
-   - sizeof(struct rte_ipv4_hdr)));
+   - ((ipv4_hdr->version_ihl & 0xf) * 4)));
}
return rte_raw_cksum(&psd_hdr, sizeof(psd_hdr));
 }
@@ -319,8 +319,8 @@ rte_ipv4_phdr_cksum(const struct rte_ipv4_hdr *ipv4_hdr, 
uint64_t ol_flags)
 /**
  * Process the IPv4 UDP or TCP checksum.
  *
- * The IPv4 header should not contains options. The IP and layer 4
- * checksum must be set to 0 in the packet by the caller.
+ * The IP and layer 4 checksum must be set to 0 in the packet by
+ * the caller.
  *
  * @param ipv4_hdr
  *   The pointer to the contiguous IPv4 header.
@@ -339,7 +339,7 @@ rte_ipv4_udptcp_cksum(const struct rte_ipv4_hdr *ipv4_hdr, 
const void *l4_hdr)
if (l3_len < sizeof(struct rte_ipv4_hdr))
return 0;
 
-   l4_len = l3_len - sizeof(struct rte_ipv4_hdr);
+   l4_len = l3_len - ((ipv4_hdr->version_ihl & 0xf) * 4);
 
cksum = rte_raw_cksum(l4_hdr, l4_len);
cksum += rte_ipv4_phdr_cksum(ipv4_hdr, 0);
-- 
2.28.0

Re: [dpdk-dev] [PATCH v5 3/9] usertools/dpdk-pmdinfo: support python3 only

[Neil Horman]

On Fri, Aug 21, 2020 at 10:14:46AM +0100, Louise Kilheeney wrote:
> Changed script to explicitly use python3 only to avoid
> maintaining python 2 and removed deprecation notice.
> 
> Cc: Neil Horman 
> 
> Signed-off-by: Louise Kilheeney 
> Reviewed-by: Bruce Richardson 
> ---
>  usertools/dpdk-pmdinfo.py | 7 +--
>  1 file changed, 1 insertion(+), 6 deletions(-)
> 
> diff --git a/usertools/dpdk-pmdinfo.py b/usertools/dpdk-pmdinfo.py
> index f9ed755176..1661982791 100755
> --- a/usertools/dpdk-pmdinfo.py
> +++ b/usertools/dpdk-pmdinfo.py
> @@ -1,4 +1,4 @@
> -#!/usr/bin/env python
> +#!/usr/bin/env python3
>  # SPDX-License-Identifier: BSD-3-Clause
>  # Copyright(c) 2016  Neil Horman 
>  
> @@ -7,8 +7,6 @@
>  # Utility to dump PMD_INFO_STRING support from an object file
>  #
>  # -
> -from __future__ import print_function
> -from __future__ import unicode_literals
>  import json
>  import io
>  import os
> @@ -28,9 +26,6 @@
>  pcidb = None
>  
>  # ===
> -if sys.version_info.major < 3:
> -print("WARNING: Python 2 is deprecated for use in DPDK, and will not 
> work in future releases.", file=sys.stderr)
> -print("Please use Python 3 instead", file=sys.stderr)
>  
>  class Vendor:
>  """
> -- 
> 2.17.1
> 
> 
Acked-by: Neil Horman 

Re: [dpdk-dev] [PATCH v5 7/9] devtools: support python3 only

[Neil Horman]

On Fri, Aug 21, 2020 at 10:14:50AM +0100, Louise Kilheeney wrote:
> Changed script to explicitly use python3 only to avoid
> maintaining python 2 and removed deprecation notice.
> 
> Cc: Neil Horman 
> Cc: Ray Kinsella 
> 
> Signed-off-by: Louise Kilheeney 
> Acked-by: Ray Kinsella 
> ---
>  devtools/update_version_map_abi.py | 7 +--
>  1 file changed, 1 insertion(+), 6 deletions(-)
> 
> diff --git a/devtools/update_version_map_abi.py 
> b/devtools/update_version_map_abi.py
> index 80a61641ed..830e6c58c8 100755
> --- a/devtools/update_version_map_abi.py
> +++ b/devtools/update_version_map_abi.py
> @@ -1,4 +1,4 @@
> -#!/usr/bin/env python
> +#!/usr/bin/env python3
>  # SPDX-License-Identifier: BSD-3-Clause
>  # Copyright(c) 2019 Intel Corporation
>  
> @@ -9,7 +9,6 @@
>  from the devtools/update-abi.sh utility.
>  """
>  
> -from __future__ import print_function
>  import argparse
>  import sys
>  import re
> @@ -160,10 +159,6 @@ def __generate_internal_abi(f_out, lines):
>  print("};", file=f_out)
>  
>  def __main():
> -if sys.version_info.major < 3:
> -print("WARNING: Python 2 is deprecated for use in DPDK, and will not 
> work in future releases.", file=sys.stderr)
> -print("Please use Python 3 instead", file=sys.stderr)
> -
>  arg_parser = argparse.ArgumentParser(
>  description='Merge versions in linker version script.')
>  
> -- 
> 2.17.1
> 
> 
Acked-by: Neil Horman 

[dpdk-dev] [PATCH v8 0/3] add basic ethdev stats with data object recursion

[Ciara Power]

v8: Rebased onto main.
v7:
  - Simplified connecting to socket by removing use of glob.
  - Fixed buffer overflow issue when reading from socket.
  - Split expected response strings over multiple lines.
v6:
  - Fixed FreeBSD build failure for unit tests.
  - Added comments and expanded commit log.
  - Add loop to call test cases stored in a list.
v5: Added unit tests for telemetry data to JSON conversion.
v4: Added missing param description to Doxygen comment.
v3:
  - Modified commit logs.
  - Changed function names to reference container.
  - Modified Doxygen comments to reference container.
v2:
  - Added support for arrays to have container values.
  - Added support for int and string arrays within dict/array.
  - Added APIs for internal container memory management.

This patchset adds support for basic ethdev statistics in Telemetry.
To do this, recursive data object support is needed to report the queue
statistics in a list. With this patch, an array or dictionary supports
uint64_t, int or string array type values, which is used for the ethdev
queue stats.

Ciara Power (2):
  telemetry: support array values in data objects
  ethdev: add common stats for telemetry

Louise Kilheeney (1):
  test/test_telemetry_data: add unit tests for data to JSON

 app/test/Makefile |   1 +
 app/test/meson.build  |   5 +-
 app/test/test_telemetry_data.c| 369 ++
 lib/librte_ethdev/rte_ethdev.c|  53 +++
 lib/librte_telemetry/rte_telemetry.h  |  70 
 .../rte_telemetry_version.map |   4 +
 lib/librte_telemetry/telemetry.c  |  56 +++
 lib/librte_telemetry/telemetry_data.c |  51 +++
 lib/librte_telemetry/telemetry_data.h |   7 +
 lib/librte_telemetry/telemetry_json.h |  33 ++
 10 files changed, 647 insertions(+), 2 deletions(-)
 create mode 100644 app/test/test_telemetry_data.c

-- 
2.17.1

[dpdk-dev] [PATCH v8 1/3] telemetry: support array values in data objects

[Ciara Power]

Arrays of type uint64_t/int/string can now be included within an array
or dict. One level of embedded containers is supported. This is
necessary to allow for instances such as the ethdev queue stats to be
reported as a list of uint64_t values, rather than having multiple dict
entries with one uint64_t value for each queue stat.

The memory management APIs provided by telemetry simplify the memory
allocation/free aspect of the embedded container. The rte_tel_data_alloc
function is called in the library/app callback to return a pointer to a
container that has been allocated memory. When adding this container
to an array/dict, a parameter is passed to indicate if the memory
should be freed by telemetry after use. This will allow reuse of the
allocated memory if the library/app wishes to do so.

Signed-off-by: Ciara Power 
Acked-by: Bruce Richardson 
---
 lib/librte_telemetry/rte_telemetry.h  | 70 +++
 .../rte_telemetry_version.map |  4 ++
 lib/librte_telemetry/telemetry.c  | 56 +++
 lib/librte_telemetry/telemetry_data.c | 51 ++
 lib/librte_telemetry/telemetry_data.h |  7 ++
 lib/librte_telemetry/telemetry_json.h | 33 +
 6 files changed, 221 insertions(+)

diff --git a/lib/librte_telemetry/rte_telemetry.h 
b/lib/librte_telemetry/rte_telemetry.h
index d13010b8fb..c165412260 100644
--- a/lib/librte_telemetry/rte_telemetry.h
+++ b/lib/librte_telemetry/rte_telemetry.h
@@ -46,6 +46,7 @@ enum rte_tel_value_type {
RTE_TEL_STRING_VAL, /** a string value */
RTE_TEL_INT_VAL,/** a signed 32-bit int value */
RTE_TEL_U64_VAL,/** an unsigned 64-bit int value */
+   RTE_TEL_CONTAINER, /** a container struct */
 };
 
 /**
@@ -136,6 +137,28 @@ __rte_experimental
 int
 rte_tel_data_add_array_u64(struct rte_tel_data *d, uint64_t x);
 
+/**
+ * Add a container to an array. A container is an existing telemetry data
+ * array. The array the container is to be added to must have been started by
+ * rte_tel_data_start_array() with RTE_TEL_CONTAINER as the type parameter.
+ * The container type must be an array of type uint64_t/int/string.
+ *
+ * @param d
+ *   The data structure passed to the callback
+ * @param val
+ *   The pointer to the container to be stored in the array.
+ * @param keep
+ *   Flag to indicate that the container memory should not be automatically
+ *   freed by the telemetry library once it has finished with the data.
+ *   1 = keep, 0 = free.
+ * @return
+ *   0 on success, negative errno on error
+ */
+__rte_experimental
+int
+rte_tel_data_add_array_container(struct rte_tel_data *d,
+   struct rte_tel_data *val, int keep);
+
 /**
  * Add a string value to a dictionary.
  * The dict must have been started by rte_tel_data_start_dict().
@@ -190,6 +213,30 @@ int
 rte_tel_data_add_dict_u64(struct rte_tel_data *d,
const char *name, uint64_t val);
 
+/**
+ * Add a container to a dictionary. A container is an existing telemetry data
+ * array. The dict the container is to be added to must have been started by
+ * rte_tel_data_start_dict(). The container must be an array of type
+ * uint64_t/int/string.
+ *
+ * @param d
+ *   The data structure passed to the callback
+ * @param name
+ *   The name the value is to be stored under in the dict.
+ * @param val
+ *   The pointer to the container to be stored in the dict.
+ * @param keep
+ *   Flag to indicate that the container memory should not be automatically
+ *   freed by the telemetry library once it has finished with the data.
+ *   1 = keep, 0 = free.
+ * @return
+ *   0 on success, negative errno on error
+ */
+__rte_experimental
+int
+rte_tel_data_add_dict_container(struct rte_tel_data *d, const char *name,
+   struct rte_tel_data *val, int keep);
+
 /**
  * This telemetry callback is used when registering a telemetry command.
  * It handles getting and formatting information to be returned to telemetry
@@ -264,4 +311,27 @@ int
 rte_telemetry_init(const char *runtime_dir, rte_cpuset_t *cpuset,
const char **err_str);
 
+/**
+ * Get a pointer to a container with memory allocated. The container is to be
+ * used embedded within an existing telemetry dict/array.
+ *
+ * @return
+ *  Pointer to a container.
+ */
+__rte_experimental
+struct rte_tel_data *
+rte_tel_data_alloc(void);
+
+/**
+ * @internal
+ * Free a container that has memory allocated.
+ *
+ * @param data
+ *  Pointer to container.
+ *.
+ */
+__rte_experimental
+void
+rte_tel_data_free(struct rte_tel_data *data);
+
 #endif
diff --git a/lib/librte_telemetry/rte_telemetry_version.map 
b/lib/librte_telemetry/rte_telemetry_version.map
index 86433c21d0..d1dbf8d586 100644
--- a/lib/librte_telemetry/rte_telemetry_version.map
+++ b/lib/librte_telemetry/rte_telemetry_version.map
@@ -1,12 +1,16 @@
 EXPERIMENTAL {
global:
 
+   rte_tel_data_add_array_container;
rte_tel_data_add_array_int;
rte

[dpdk-dev] [PATCH v8 3/3] ethdev: add common stats for telemetry

[Ciara Power]

The ethdev library now registers a telemetry command for common ethdev
statistics.

An example usage is shown below:

Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
{"version": "DPDK 20.08.0-rc1", "pid": 14119, "max_output_len": 16384}
--> /ethdev/stats,0
{"/ethdev/stats": {"ipackets": 0, "opackets": 0, "ibytes": 0, "obytes": \
0, "imissed": 0, "ierrors": 0, "oerrors": 0, "rx_nombuf": 0, \
"q_ipackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], \
"q_opackets": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], \
"q_ibytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], \
"q_obytes": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], \
"q_errors": [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]}}

Signed-off-by: Ciara Power 
Acked-by: Bruce Richardson 
---
 lib/librte_ethdev/rte_ethdev.c | 53 ++
 1 file changed, 53 insertions(+)

diff --git a/lib/librte_ethdev/rte_ethdev.c b/lib/librte_ethdev/rte_ethdev.c
index 7858ad5f11..8ee14b1b54 100644
--- a/lib/librte_ethdev/rte_ethdev.c
+++ b/lib/librte_ethdev/rte_ethdev.c
@@ -5275,6 +5275,57 @@ handle_port_list(const char *cmd __rte_unused,
return 0;
 }
 
+static void
+add_port_queue_stats(struct rte_tel_data *d, uint64_t *q_stats,
+   const char *stat_name)
+{
+   int q;
+   struct rte_tel_data *q_data = rte_tel_data_alloc();
+   rte_tel_data_start_array(q_data, RTE_TEL_U64_VAL);
+   for (q = 0; q < RTE_ETHDEV_QUEUE_STAT_CNTRS; q++)
+   rte_tel_data_add_array_u64(q_data, q_stats[q]);
+   rte_tel_data_add_dict_container(d, stat_name, q_data, 0);
+}
+
+#define ADD_DICT_STAT(stats, s) rte_tel_data_add_dict_u64(d, #s, stats.s)
+
+static int
+handle_port_stats(const char *cmd __rte_unused,
+   const char *params,
+   struct rte_tel_data *d)
+{
+   struct rte_eth_stats stats;
+   int port_id, ret;
+
+   if (params == NULL || strlen(params) == 0 || !isdigit(*params))
+   return -1;
+
+   port_id = atoi(params);
+   if (!rte_eth_dev_is_valid_port(port_id))
+   return -1;
+
+   ret = rte_eth_stats_get(port_id, &stats);
+   if (ret < 0)
+   return -1;
+
+   rte_tel_data_start_dict(d);
+   ADD_DICT_STAT(stats, ipackets);
+   ADD_DICT_STAT(stats, opackets);
+   ADD_DICT_STAT(stats, ibytes);
+   ADD_DICT_STAT(stats, obytes);
+   ADD_DICT_STAT(stats, imissed);
+   ADD_DICT_STAT(stats, ierrors);
+   ADD_DICT_STAT(stats, oerrors);
+   ADD_DICT_STAT(stats, rx_nombuf);
+   add_port_queue_stats(d, stats.q_ipackets, "q_ipackets");
+   add_port_queue_stats(d, stats.q_opackets, "q_opackets");
+   add_port_queue_stats(d, stats.q_ibytes, "q_ibytes");
+   add_port_queue_stats(d, stats.q_obytes, "q_obytes");
+   add_port_queue_stats(d, stats.q_errors, "q_errors");
+
+   return 0;
+}
+
 static int
 handle_port_xstats(const char *cmd __rte_unused,
const char *params,
@@ -5361,6 +5412,8 @@ RTE_INIT(ethdev_init_telemetry)
 {
rte_telemetry_register_cmd("/ethdev/list", handle_port_list,
"Returns list of available ethdev ports. Takes no 
parameters");
+   rte_telemetry_register_cmd("/ethdev/stats", handle_port_stats,
+   "Returns the common stats for a port. Parameters: int 
port_id");
rte_telemetry_register_cmd("/ethdev/xstats", handle_port_xstats,
"Returns the extended stats for a port. Parameters: int 
port_id");
rte_telemetry_register_cmd("/ethdev/link_status",
-- 
2.17.1

[dpdk-dev] [PATCH v8 2/3] test/test_telemetry_data: add unit tests for data to JSON

[Ciara Power]

From: Louise Kilheeney 

This patch adds tests for verifying telemetry data structures are
converted to JSON as expected. Both flat and recursive data structures
are tested, for all possible value types.

The app connects to the telemetry socket as a client, and registers one
command with a corresponding callback function. Each time the callback
function is called, it copies a global data variable to the data pointer
passed in by telemetry.
When a test case is run, the test case function builds up the global
data variable with the relevant data types, and the expected json string
output which should be generated from that. The 'test_output()' function
is used to trigger the callback and ensure the actual output matches
that expected.

Signed-off-by: Louise Kilheeney 
Signed-off-by: Ciara Power 
Acked-by: Bruce Richardson 
---
 app/test/Makefile  |   1 +
 app/test/meson.build   |   5 +-
 app/test/test_telemetry_data.c | 369 +
 3 files changed, 373 insertions(+), 2 deletions(-)
 create mode 100644 app/test/test_telemetry_data.c

diff --git a/app/test/Makefile b/app/test/Makefile
index f4065271e4..1cb64089c1 100644
--- a/app/test/Makefile
+++ b/app/test/Makefile
@@ -145,6 +145,7 @@ SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6.c
 SRCS-$(CONFIG_RTE_LIBRTE_LPM) += test_lpm6_perf.c
 
 SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += test_telemetry_json.c
+SRCS-$(CONFIG_RTE_LIBRTE_TELEMETRY) += test_telemetry_data.c
 
 SRCS-y += test_debug.c
 SRCS-y += test_errno.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 786a213972..4a72fe5b61 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -170,6 +170,7 @@ test_deps = ['acl',
'ring',
'security',
'stack',
+   'telemetry',
'timer'
 ]
 
@@ -345,8 +346,8 @@ if dpdk_conf.has('RTE_LIBRTE_SKELETON_EVENTDEV_PMD')
test_deps += 'pmd_skeleton_event'
 endif
 if dpdk_conf.has('RTE_LIBRTE_TELEMETRY')
-   test_sources += 'test_telemetry_json.c'
-   fast_tests += [['telemetry_json_autotest', true]]
+   test_sources += ['test_telemetry_json.c', 'test_telemetry_data.c']
+   fast_tests += [['telemetry_json_autotest', true], 
['telemetry_data_autotest', true]]
 endif
 
 # The following linkages of drivers are required because
diff --git a/app/test/test_telemetry_data.c b/app/test/test_telemetry_data.c
new file mode 100644
index 00..7a31e68a78
--- /dev/null
+++ b/app/test/test_telemetry_data.c
@@ -0,0 +1,369 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright 2020 Intel Corporation
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include 
+#include 
+#include 
+#include 
+
+#include "test.h"
+#include "telemetry_data.h"
+
+#define TELEMETRY_VERSION "v2"
+#define REQUEST_CMD "/test"
+#define BUF_SIZE 1024
+#define TEST_OUTPUT(exp) test_output(__func__, exp)
+
+static struct rte_tel_data response_data;
+static int sock;
+
+/*
+ * This function is the callback registered with Telemetry to be used when
+ * the /test command is requested. This callback returns the global data built
+ * up by the individual test cases.
+ */
+static int
+test_cb(const char *cmd __rte_unused, const char *params __rte_unused,
+   struct rte_tel_data *d)
+{
+   *d = response_data;
+   return 0;
+}
+
+/*
+ * This function is called by each test case function. It communicates with
+ * the telemetry socket by requesting the /test command, and reading the
+ * response. The expected response is passed in by the test case function,
+ * and is compared to the actual response received from Telemetry.
+ */
+static int
+test_output(const char *func_name, const char *expected)
+{
+   int bytes;
+   char buf[BUF_SIZE * 16];
+   if (write(sock, REQUEST_CMD, strlen(REQUEST_CMD)) < 0) {
+   printf("%s: Error with socket write - %s\n", __func__,
+   strerror(errno));
+   return -1;
+   }
+   bytes = read(sock, buf, sizeof(buf) - 1);
+   if (bytes < 0) {
+   printf("%s: Error with socket read - %s\n", __func__,
+   strerror(errno));
+   return -1;
+   }
+   buf[bytes] = '\0';
+   printf("%s: buf = '%s', expected = '%s'\n", func_name, buf, expected);
+   return strncmp(expected, buf, sizeof(buf));
+}
+
+static int
+test_dict_with_array_int_values(void)
+{
+   int i;
+
+   struct rte_tel_data *child_data = rte_tel_data_alloc();
+   rte_tel_data_start_array(child_data, RTE_TEL_INT_VAL);
+
+   struct rte_tel_data *child_data2 = rte_tel_data_alloc();
+   rte_tel_data_start_array(child_data2, RTE_TEL_INT_VAL);
+
+   memset(&response_data, 0, sizeof(response_data));
+   rte_tel_data_start_dict(&response_data);
+
+   for (i = 0; i < 5; i++) {
+   rte_tel_data_add_array_int(child_data, i);
+   rte_tel_data_add_array_int(child_data2, i);
+   }
+
+   rte_tel_data

Re: [dpdk-dev] [PATCH v2 20/37] doc: remove references to make in bbdev guides

[Chautru, Nicolas]

> From: Power, Ciara 
> Hi Nicolas,
> 
> 
> >-Original Message-
> >From: Chautru, Nicolas 
> >Sent: Friday 21 August 2020 03:13
> >To: Power, Ciara ; dev@dpdk.org
> >Cc: Mcnamara, John ; Kovacevic, Marko
> >
> >Subject: RE: [PATCH v2 20/37] doc: remove references to make in bbdev
> >guides
> >
> >> From: Power, Ciara  Make is no longer
> >> supported for compiling DPDK, references are now removed in the
> documentation.
> >>
> >> Reviewed-by: Kevin Laatz 
> >> Signed-off-by: Ciara Power 
> >> ---
> >>  doc/guides/bbdevs/fpga_5gnr_fec.rst |  7 ++-
> >> doc/guides/bbdevs/fpga_lte_fec.rst  |  7 ++-
> >>  doc/guides/bbdevs/turbo_sw.rst  | 15 ---
> >>  3 files changed, 4 insertions(+), 25 deletions(-)
> >>
> 
> >>cd 
> >> -  insmod ./build/kmod/igb_uio.ko
> >> +  insmod .//kernel/linux/igb_uio/igb_uio.ko
> >
> >Hi,
> >Should this more specific? What do you mean by  here? Is
> >this defined somewhere else explicitly in doc?
> >In other commit (with the python script) the assumption is that meson
> >will install under /build directory with hard
> >coded path, ie. not forcing to other location.
> >Can you clarify so that we are comprehensive and avoid confusion?
> >Thanks
> >Nic
> >
> >
> >
> 
> The "" here represents the meson build folder, which is named by
> the user so can vary.
> In the python script, I added "build" as the hardcoded path for the build 
> folder,
> based on the example in the prog_guide/build-sdk-meson guide, which chooses
> to name the folder "build".

Yes and I find this discrepancy as a source of confusion really. Either build 
location is flexible or not and in any case this needs to be deterministic and 
solid. 
I am not an expert on meson, unsure what is the best way to handle this but I 
certainly see room for errors we did not have with make. 
Any thought?

> 
> Thanks,
> Ciara

Re: [dpdk-dev] [PATCH v2] sched: fix port time rounding error

[Kinsella, Ray]

On 20/08/2020 15:32, Kevin Traynor wrote:
> Hi,
> 
> On 25/06/2020 10:59, alangordonde...@gmail.com wrote:
>> From: Alan Dewar 
>>
>> The QoS scheduler works off port time that is computed from the number
>> of CPU cycles that have elapsed since the last time the port was
>> polled.   It divides the number of elapsed cycles to calculate how
>> many bytes can be sent, however this division can generate rounding
>> errors, where some fraction of a byte sent may be lost.
>>
>> Lose enough of these fractional bytes and the QoS scheduler
>> underperforms.  The problem is worse with low bandwidths.
>>
>> To compensate for this rounding error this fix doesn't advance the
>> port's time_cpu_cycles by the number of cycles that have elapsed,
>> but by multiplying the computed number of bytes that can be sent
>> (which has been rounded down) by number of cycles per byte.
>> This will mean that port's time_cpu_cycles will lag behind the CPU
>> cycles momentarily.  At the next poll, the lag will be taken into
>> account.
>>
>> v2:
>> If the cycles value wraps (100 year+) reset the port's cpu cycle back
>> to zero.
>>
>> Fixes: de3cfa2c98 ("sched: initial import")
>>
>> Signed-off-by: Alan Dewar 
>> ---
>>  lib/librte_sched/rte_sched.c | 11 +--
>>  1 file changed, 9 insertions(+), 2 deletions(-)
>>
>> diff --git a/lib/librte_sched/rte_sched.c b/lib/librte_sched/rte_sched.c
>> index c0983ddda..7c022cd61 100644
>> --- a/lib/librte_sched/rte_sched.c
>> +++ b/lib/librte_sched/rte_sched.c
>> @@ -222,6 +222,7 @@ struct rte_sched_port {
>>  uint64_t time_cpu_bytes;  /* Current CPU time measured in bytes */
>>  uint64_t time;/* Current NIC TX time measured in bytes 
>> */
>>  struct rte_reciprocal inv_cycles_per_byte; /* CPU cycles per byte */
>> +uint64_t cycles_per_byte;
>>  
> 
> I was backporting this patch to 18.11. The older ABI checker complains
> about this structure change.
> 
> "cycles_per_byte has been added at the middle position of this
> structural type."
> 
> Isn't this an ABI break? Dropping from 18.11 for time being.

So it looks like rte_sched_port * is an opaque pointers as it's contents are 
only
known to rte_sched.c and not outside. To everyone else it is an opaque data 
structure, 
so structural changes to it would not be an ABI breakage.

> 
>>  /* Grinders */
>>  struct rte_mbuf **pkts_out;
>> @@ -852,6 +853,7 @@ rte_sched_port_config(struct rte_sched_port_params 
>> *params)
>>  cycles_per_byte = (rte_get_tsc_hz() << RTE_SCHED_TIME_SHIFT)
>>  / params->rate;
>>  port->inv_cycles_per_byte = rte_reciprocal_value(cycles_per_byte);
>> +port->cycles_per_byte = cycles_per_byte;
>>  
>>  /* Grinders */
>>  port->pkts_out = NULL;
>> @@ -2673,16 +2675,21 @@ static inline void
>>  rte_sched_port_time_resync(struct rte_sched_port *port)
>>  {
>>  uint64_t cycles = rte_get_tsc_cycles();
>> -uint64_t cycles_diff = cycles - port->time_cpu_cycles;
>> +uint64_t cycles_diff;
>>  uint64_t bytes_diff;
>>  uint32_t i;
>>  
>> +if (cycles < port->time_cpu_cycles)
>> +port->time_cpu_cycles = 0;
>> +
>> +cycles_diff = cycles - port->time_cpu_cycles;
>>  /* Compute elapsed time in bytes */
>>  bytes_diff = rte_reciprocal_divide(cycles_diff << RTE_SCHED_TIME_SHIFT,
>> port->inv_cycles_per_byte);
>>  
>>  /* Advance port time */
>> -port->time_cpu_cycles = cycles;
>> +port->time_cpu_cycles +=
>> +(bytes_diff * port->cycles_per_byte) >> RTE_SCHED_TIME_SHIFT;
>>  port->time_cpu_bytes += bytes_diff;
>>  if (port->time < port->time_cpu_bytes)
>>  port->time = port->time_cpu_bytes;
>>
> 

[dpdk-dev] [PATCH 1/4] raw/ioat: support multiple devices being tested

[Bruce Richardson]

The current selftest function uses a single global variable to track state
which implies that only a single instance can have the selftest function
called on it. Change this to an array to allow multiple instances to be
tested.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/ioat_rawdev_test.c | 17 +
 1 file changed, 13 insertions(+), 4 deletions(-)

diff --git a/drivers/raw/ioat/ioat_rawdev_test.c 
b/drivers/raw/ioat/ioat_rawdev_test.c
index 724cdb9e74..6262995256 100644
--- a/drivers/raw/ioat/ioat_rawdev_test.c
+++ b/drivers/raw/ioat/ioat_rawdev_test.c
@@ -9,10 +9,13 @@
 #include "rte_ioat_rawdev.h"
 #include "ioat_private.h"
 
+#define MAX_SUPPORTED_RAWDEVS 64
+#define TEST_SKIPPED 77
+
 int ioat_rawdev_test(uint16_t dev_id); /* pre-define to keep compiler happy */
 
 static struct rte_mempool *pool;
-static unsigned short expected_ring_size;
+static unsigned short expected_ring_size[MAX_SUPPORTED_RAWDEVS];
 
 static int
 test_enqueue_copies(int dev_id)
@@ -149,10 +152,16 @@ ioat_rawdev_test(uint16_t dev_id)
unsigned int nb_xstats;
unsigned int i;
 
+   if (dev_id >= MAX_SUPPORTED_RAWDEVS) {
+   printf("Skipping test. Cannot test rawdevs with id's greater 
than %d\n",
+   MAX_SUPPORTED_RAWDEVS);
+   return TEST_SKIPPED;
+   }
+
rte_rawdev_info_get(dev_id, &info, sizeof(p));
-   if (p.ring_size != expected_ring_size) {
+   if (p.ring_size != expected_ring_size[dev_id]) {
printf("Error, initial ring size is not as expected (Actual: 
%d, Expected: %d)\n",
-   (int)p.ring_size, expected_ring_size);
+   (int)p.ring_size, expected_ring_size[dev_id]);
return -1;
}
 
@@ -167,7 +176,7 @@ ioat_rawdev_test(uint16_t dev_id)
IOAT_TEST_RINGSIZE, (int)p.ring_size);
return -1;
}
-   expected_ring_size = p.ring_size;
+   expected_ring_size[dev_id] = p.ring_size;
 
if (rte_rawdev_start(dev_id) != 0) {
printf("Error with rte_rawdev_start()\n");
-- 
2.25.1

[dpdk-dev] [PATCH 0/4] simplify unit-testing of rawdevs

[Bruce Richardson]

At present the "rawdev_autotest" command creates a skeleton rawdev and runs
a series of tests on the rawdev API and on that rawdev. While the rawdev
API set includes a "selftest" function, it is not hooked up to this test so
to test an individual rawdev driver, e.g. ioat, requires that a new test
command be added.

This patchset improves the situation by changing the UT to first run the
existing API tests, but then call selftest on all rawdevs on the system.
This removes the need for any new test commands for new drivers. If there
are multiple rawdevs on a system, the sub-set to be tested can be limited
via existing means such as using the device block/allow EAL parameters or
similarly via vdev args, etc.

As part of this change, the ioat rawdev autotest is fixed to allow calling
on multiple instances inside the one test run, and thereafter the custom
test command for it is removed as it is no longer necessary.

Bruce Richardson (4):
  raw/ioat: support multiple devices being tested
  raw/ioat: include extra info in error messages
  app/test: change rawdev autotest to run selftest on all devs
  app/test: remove ioat-specific autotest

 app/test/test_rawdev.c  | 38 +---
 drivers/raw/ioat/ioat_rawdev_test.c | 69 -
 2 files changed, 68 insertions(+), 39 deletions(-)

-- 
2.25.1

[dpdk-dev] [PATCH 2/4] raw/ioat: include extra info in error messages

[Bruce Richardson]

In case of any failures, include the function name and the line number of
the error message in the message, to make tracking down the failure easier.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/ioat_rawdev_test.c | 52 +++--
 1 file changed, 34 insertions(+), 18 deletions(-)

diff --git a/drivers/raw/ioat/ioat_rawdev_test.c 
b/drivers/raw/ioat/ioat_rawdev_test.c
index 6262995256..534b07b124 100644
--- a/drivers/raw/ioat/ioat_rawdev_test.c
+++ b/drivers/raw/ioat/ioat_rawdev_test.c
@@ -17,6 +17,22 @@ int ioat_rawdev_test(uint16_t dev_id); /* pre-define to keep 
compiler happy */
 static struct rte_mempool *pool;
 static unsigned short expected_ring_size[MAX_SUPPORTED_RAWDEVS];
 
+#define PRINT_ERR(...) print_err(__func__, __LINE__, __VA_ARGS__)
+
+static inline int
+print_err(const char *func, int lineno, const char *format, ...)
+{
+   va_list ap;
+   int ret;
+
+   ret = fprintf(stderr, "In %s:%d - ", func, lineno);
+   va_start(ap, format);
+   ret += vfprintf(stderr, format, ap);
+   va_end(ap);
+
+   return ret;
+}
+
 static int
 test_enqueue_copies(int dev_id)
 {
@@ -46,7 +62,7 @@ test_enqueue_copies(int dev_id)
(uintptr_t)src,
(uintptr_t)dst,
0 /* no fence */) != 1) {
-   printf("Error with rte_ioat_enqueue_copy\n");
+   PRINT_ERR("Error with rte_ioat_enqueue_copy\n");
return -1;
}
rte_ioat_do_copies(dev_id);
@@ -54,18 +70,18 @@ test_enqueue_copies(int dev_id)
 
if (rte_ioat_completed_copies(dev_id, 1, (void *)&completed[0],
(void *)&completed[1]) != 1) {
-   printf("Error with rte_ioat_completed_copies\n");
+   PRINT_ERR("Error with rte_ioat_completed_copies\n");
return -1;
}
if (completed[0] != src || completed[1] != dst) {
-   printf("Error with completions: got (%p, %p), not 
(%p,%p)\n",
+   PRINT_ERR("Error with completions: got (%p, %p), not 
(%p,%p)\n",
completed[0], completed[1], src, dst);
return -1;
}
 
for (i = 0; i < length; i++)
if (dst_data[i] != src_data[i]) {
-   printf("Data mismatch at char %u\n", i);
+   PRINT_ERR("Data mismatch at char %u\n", i);
return -1;
}
rte_pktmbuf_free(src);
@@ -98,7 +114,7 @@ test_enqueue_copies(int dev_id)
(uintptr_t)srcs[i],
(uintptr_t)dsts[i],
0 /* nofence */) != 1) {
-   printf("Error with rte_ioat_enqueue_copy for 
buffer %u\n",
+   PRINT_ERR("Error with rte_ioat_enqueue_copy for 
buffer %u\n",
i);
return -1;
}
@@ -108,18 +124,18 @@ test_enqueue_copies(int dev_id)
 
if (rte_ioat_completed_copies(dev_id, 64, (void *)completed_src,
(void *)completed_dst) != RTE_DIM(srcs)) {
-   printf("Error with rte_ioat_completed_copies\n");
+   PRINT_ERR("Error with rte_ioat_completed_copies\n");
return -1;
}
for (i = 0; i < RTE_DIM(srcs); i++) {
char *src_data, *dst_data;
 
if (completed_src[i] != srcs[i]) {
-   printf("Error with source pointer %u\n", i);
+   PRINT_ERR("Error with source pointer %u\n", i);
return -1;
}
if (completed_dst[i] != dsts[i]) {
-   printf("Error with dest pointer %u\n", i);
+   PRINT_ERR("Error with dest pointer %u\n", i);
return -1;
}
 
@@ -127,7 +143,7 @@ test_enqueue_copies(int dev_id)
dst_data = rte_pktmbuf_mtod(dsts[i], char *);
for (j = 0; j < length; j++)
if (src_data[j] != dst_data[j]) {
-   printf("Error with copy of packet %u, 
byte %u\n",
+   PRINT_ERR("Error with copy of packet 
%u, byte %u\n",
i, j);
return -1;
}
@@ -160,26 +176,26 @@ ioat_rawdev_test(uint16_t dev_id)
 
   

[dpdk-dev] [PATCH 3/4] app/test: change rawdev autotest to run selftest on all devs

[Bruce Richardson]

Rather than having each rawdev provide its own autotest command, we can
instead just use the generic rawdev_autotest to test any and all available
rawdevs.

Signed-off-by: Bruce Richardson 
---
 app/test/test_rawdev.c | 28 ++--
 1 file changed, 26 insertions(+), 2 deletions(-)

diff --git a/app/test/test_rawdev.c b/app/test/test_rawdev.c
index d8d9595be1..0e25ccf8dc 100644
--- a/app/test/test_rawdev.c
+++ b/app/test/test_rawdev.c
@@ -14,14 +14,38 @@
 static int
 test_rawdev_selftest_impl(const char *pmd, const char *opts)
 {
+   int ret;
+
+   printf("\n### Test rawdev infrastructure using skeleton driver\n");
rte_vdev_init(pmd, opts);
-   return rte_rawdev_selftest(rte_rawdev_get_dev_id(pmd));
+   ret = rte_rawdev_selftest(rte_rawdev_get_dev_id(pmd));
+   rte_vdev_uninit(pmd);
+   return ret;
 }
 
 static int
 test_rawdev_selftest_skeleton(void)
 {
-   return test_rawdev_selftest_impl("rawdev_skeleton", "");
+   const int count = rte_rawdev_count();
+   int ret = 0;
+   int i;
+
+   /* basic sanity on rawdev infrastructure */
+   if (test_rawdev_selftest_impl("rawdev_skeleton", "") < 0)
+   return -1;
+
+   /* now run self-test on all rawdevs */
+   if (count > 0)
+   printf("\n### Run selftest on each available rawdev\n");
+   for (i = 0; i < count; i++) {
+   int result = rte_rawdev_selftest(i);
+   printf("Rawdev %u (%s) selftest: %s\n", i,
+   rte_rawdevs[i].name,
+   result == 0 ? "Passed" : "Failed");
+   ret |=  result;
+   }
+
+   return ret;
 }
 
 REGISTER_TEST_COMMAND(rawdev_autotest, test_rawdev_selftest_skeleton);
-- 
2.25.1

[dpdk-dev] [PATCH 4/4] app/test: remove ioat-specific autotest

[Bruce Richardson]

Since the rawdev autotest can now be used to test all rawdevs on the
system, there is no need for a dedicated ioat autotest command.

Signed-off-by: Bruce Richardson 
---
 app/test/test_rawdev.c | 20 
 1 file changed, 20 deletions(-)

diff --git a/app/test/test_rawdev.c b/app/test/test_rawdev.c
index 0e25ccf8dc..44651ba3cc 100644
--- a/app/test/test_rawdev.c
+++ b/app/test/test_rawdev.c
@@ -49,23 +49,3 @@ test_rawdev_selftest_skeleton(void)
 }
 
 REGISTER_TEST_COMMAND(rawdev_autotest, test_rawdev_selftest_skeleton);
-
-static int
-test_rawdev_selftest_ioat(void)
-{
-   const int count = rte_rawdev_count();
-   int i;
-
-   for (i = 0; i < count; i++) {
-   struct rte_rawdev_info info = { .dev_private = NULL };
-   if (rte_rawdev_info_get(i, &info, 0) == 0 &&
-   strstr(info.driver_name, "ioat") != NULL)
-   return rte_rawdev_selftest(i) == 0 ?
-   TEST_SUCCESS : TEST_FAILED;
-   }
-
-   printf("No IOAT rawdev found, skipping tests\n");
-   return TEST_SKIPPED;
-}
-
-REGISTER_TEST_COMMAND(ioat_rawdev_autotest, test_rawdev_selftest_ioat);
-- 
2.25.1

[dpdk-dev] [PATCH v2 01/18] raw/ioat: add a flag to control copying handle parameters

[Bruce Richardson]

From: Cheng Jiang 

Add a flag which controls whether rte_ioat_enqueue_copy and
rte_ioat_completed_copies function should process handle parameters. Not
doing so can improve the performance when handle parameters are not
necessary.

Signed-off-by: Cheng Jiang 
Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/ioat_rawdev.c |  2 ++
 drivers/raw/ioat/rte_ioat_rawdev.h | 45 ++
 2 files changed, 36 insertions(+), 11 deletions(-)

diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c
index 7f1a154360..856b55cb6e 100644
--- a/drivers/raw/ioat/ioat_rawdev.c
+++ b/drivers/raw/ioat/ioat_rawdev.c
@@ -58,6 +58,7 @@ ioat_dev_configure(const struct rte_rawdev *dev, 
rte_rawdev_obj_t config,
return -EINVAL;
 
ioat->ring_size = params->ring_size;
+   ioat->hdls_disable = params->hdls_disable;
if (ioat->desc_ring != NULL) {
rte_memzone_free(ioat->desc_mz);
ioat->desc_ring = NULL;
@@ -122,6 +123,7 @@ ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t 
dev_info,
return -EINVAL;
 
cfg->ring_size = ioat->ring_size;
+   cfg->hdls_disable = ioat->hdls_disable;
return 0;
 }
 
diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h 
b/drivers/raw/ioat/rte_ioat_rawdev.h
index f765a65571..4bc6491d91 100644
--- a/drivers/raw/ioat/rte_ioat_rawdev.h
+++ b/drivers/raw/ioat/rte_ioat_rawdev.h
@@ -34,7 +34,8 @@
  * an ioat rawdev instance.
  */
 struct rte_ioat_rawdev_config {
-   unsigned short ring_size;
+   unsigned short ring_size; /**< size of job submission descriptor ring */
+   bool hdls_disable;/**< if set, ignore user-supplied handle params */
 };
 
 /**
@@ -52,6 +53,7 @@ struct rte_ioat_rawdev {
 
unsigned short ring_size;
struct rte_ioat_generic_hw_desc *desc_ring;
+   bool hdls_disable;
__m128i *hdls; /* completion handles for returning to user */
 
 
@@ -84,10 +86,14 @@ struct rte_ioat_rawdev {
  *   The length of the data to be copied
  * @param src_hdl
  *   An opaque handle for the source data, to be returned when this operation
- *   has been completed and the user polls for the completion details
+ *   has been completed and the user polls for the completion details.
+ *   NOTE: If hdls_disable configuration option for the device is set, this
+ *   parameter is ignored.
  * @param dst_hdl
  *   An opaque handle for the destination data, to be returned when this
- *   operation has been completed and the user polls for the completion details
+ *   operation has been completed and the user polls for the completion 
details.
+ *   NOTE: If hdls_disable configuration option for the device is set, this
+ *   parameter is ignored.
  * @param fence
  *   A flag parameter indicating that hardware should not begin to perform any
  *   subsequently enqueued copy operations until after this operation has
@@ -121,8 +127,10 @@ rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, 
phys_addr_t dst,
desc->u.control_raw = (uint32_t)((!!fence << 4) | (!(write & 0xF)) << 
3);
desc->src_addr = src;
desc->dest_addr = dst;
+   if (!ioat->hdls_disable)
+   ioat->hdls[write] = _mm_set_epi64x((int64_t)dst_hdl,
+   (int64_t)src_hdl);
 
-   ioat->hdls[write] = _mm_set_epi64x((int64_t)dst_hdl, (int64_t)src_hdl);
rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]);
 
ioat->enqueued++;
@@ -168,19 +176,29 @@ rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, 
int *error)
 /**
  * Returns details of copy operations that have been completed
  *
- * Returns to the caller the user-provided "handles" for the copy operations
- * which have been completed by the hardware, and not already returned by
- * a previous call to this API.
+ * If the hdls_disable option was not set when the device was configured,
+ * the function will return to the caller the user-provided "handles" for
+ * the copy operations which have been completed by the hardware, and not
+ * already returned by a previous call to this API.
+ * If the hdls_disable option for the device was set on configure, the
+ * max_copies, src_hdls and dst_hdls parameters will be ignored, and the
+ * function returns the number of newly-completed operations.
  *
  * @param dev_id
  *   The rawdev device id of the ioat instance
  * @param max_copies
  *   The number of entries which can fit in the src_hdls and dst_hdls
- *   arrays, i.e. max number of completed operations to report
+ *   arrays, i.e. max number of completed operations to report.
+ *   NOTE: If hdls_disable configuration option for the device is set, this
+ *   parameter is ignored.
  * @param src_hdls
- *   Array to hold the source handle parameters of the completed copies
+ *   Array to hold the source handle parameters of the completed copies.
+ *   NOTE: If hdls_disable configuration option for the device is set, this
+ *   param

[dpdk-dev] [PATCH v2 02/18] raw/ioat: split header for readability

[Bruce Richardson]

Rather than having a single long complicated header file for general use we
can split things so that there is one header with all the publically needed
information - data structs and function prototypes - while the rest of the
internal details are put separately. This makes it easier to read,
understand and use the APIs.

Signed-off-by: Bruce Richardson 
---

There are a couple of checkpatch errors about spacing in this patch,
however, it appears that these are false positives.
---
 drivers/raw/ioat/meson.build   |   1 +
 drivers/raw/ioat/rte_ioat_rawdev.h | 144 +-
 drivers/raw/ioat/rte_ioat_rawdev_fns.h | 164 +
 3 files changed, 171 insertions(+), 138 deletions(-)
 create mode 100644 drivers/raw/ioat/rte_ioat_rawdev_fns.h

diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build
index 0878418aee..f66e9b605e 100644
--- a/drivers/raw/ioat/meson.build
+++ b/drivers/raw/ioat/meson.build
@@ -8,4 +8,5 @@ sources = files('ioat_rawdev.c',
 deps += ['rawdev', 'bus_pci', 'mbuf']
 
 install_headers('rte_ioat_rawdev.h',
+   'rte_ioat_rawdev_fns.h',
'rte_ioat_spec.h')
diff --git a/drivers/raw/ioat/rte_ioat_rawdev.h 
b/drivers/raw/ioat/rte_ioat_rawdev.h
index 4bc6491d91..7ace5c085a 100644
--- a/drivers/raw/ioat/rte_ioat_rawdev.h
+++ b/drivers/raw/ioat/rte_ioat_rawdev.h
@@ -14,12 +14,7 @@
  * @b EXPERIMENTAL: these structures and APIs may change without prior notice
  */
 
-#include 
-#include 
-#include 
-#include 
-#include 
-#include "rte_ioat_spec.h"
+#include 
 
 /** Name of the device driver */
 #define IOAT_PMD_RAWDEV_NAME rawdev_ioat
@@ -38,38 +33,6 @@ struct rte_ioat_rawdev_config {
bool hdls_disable;/**< if set, ignore user-supplied handle params */
 };
 
-/**
- * @internal
- * Structure representing a device instance
- */
-struct rte_ioat_rawdev {
-   struct rte_rawdev *rawdev;
-   const struct rte_memzone *mz;
-   const struct rte_memzone *desc_mz;
-
-   volatile struct rte_ioat_registers *regs;
-   phys_addr_t status_addr;
-   phys_addr_t ring_addr;
-
-   unsigned short ring_size;
-   struct rte_ioat_generic_hw_desc *desc_ring;
-   bool hdls_disable;
-   __m128i *hdls; /* completion handles for returning to user */
-
-
-   unsigned short next_read;
-   unsigned short next_write;
-
-   /* some statistics for tracking, if added/changed update xstats fns*/
-   uint64_t enqueue_failed __rte_cache_aligned;
-   uint64_t enqueued;
-   uint64_t started;
-   uint64_t completed;
-
-   /* to report completions, the device will write status back here */
-   volatile uint64_t status __rte_cache_aligned;
-};
-
 /**
  * Enqueue a copy operation onto the ioat device
  *
@@ -104,38 +67,7 @@ struct rte_ioat_rawdev {
 static inline int
 rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst,
unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl,
-   int fence)
-{
-   struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private;
-   unsigned short read = ioat->next_read;
-   unsigned short write = ioat->next_write;
-   unsigned short mask = ioat->ring_size - 1;
-   unsigned short space = mask + read - write;
-   struct rte_ioat_generic_hw_desc *desc;
-
-   if (space == 0) {
-   ioat->enqueue_failed++;
-   return 0;
-   }
-
-   ioat->next_write = write + 1;
-   write &= mask;
-
-   desc = &ioat->desc_ring[write];
-   desc->size = length;
-   /* set descriptor write-back every 16th descriptor */
-   desc->u.control_raw = (uint32_t)((!!fence << 4) | (!(write & 0xF)) << 
3);
-   desc->src_addr = src;
-   desc->dest_addr = dst;
-   if (!ioat->hdls_disable)
-   ioat->hdls[write] = _mm_set_epi64x((int64_t)dst_hdl,
-   (int64_t)src_hdl);
-
-   rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]);
-
-   ioat->enqueued++;
-   return 1;
-}
+   int fence);
 
 /**
  * Trigger hardware to begin performing enqueued copy operations
@@ -147,31 +79,7 @@ rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, 
phys_addr_t dst,
  *   The rawdev device id of the ioat instance
  */
 static inline void
-rte_ioat_do_copies(int dev_id)
-{
-   struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private;
-   ioat->desc_ring[(ioat->next_write - 1) & (ioat->ring_size - 1)].u
-   .control.completion_update = 1;
-   rte_compiler_barrier();
-   ioat->regs->dmacount = ioat->next_write;
-   ioat->started = ioat->enqueued;
-}
-
-/**
- * @internal
- * Returns the index of the last completed operation.
- */
-static inline int
-rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error)
-{
-   uint64_t status = ioat->status;
-
-   /* lower 3 bits indicate "transfer status" : active, idle, halted.
-* We can 

[dpdk-dev] [PATCH v2 00/18] raw/ioat: enhancements and new hardware support

[Bruce Richardson]

This patchset adds some small enhancements, some rework and also support
for new hardware to the ioat rawdev driver. Most rework and enhancements
are largely self-explanatory from the individual patches.

The new hardware support is for the Intel(R) DSA accelerator which will be
present in future Intel processors. A description of this new hardware is
covered in [1]. Functions specific to the new hardware use the "idxd"
prefix, for consistency with the kernel driver. *** SUBJECT HERE ***

[1] https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator

---
V2:
 * Included documentation additions in the set
 * Split off the rawdev unit test changes to a separate patchset for easier
   review
 * General code improvements and cleanups


Bruce Richardson (14):
  raw/ioat: split header for readability
  raw/ioat: make the HW register spec private
  raw/ioat: add skeleton for VFIO/UIO based DSA device
  raw/ioat: include example configuration script
  raw/ioat: create rawdev instances on idxd PCI probe
  raw/ioat: add datapath data structures for idxd devices
  raw/ioat: add configure function for idxd devices
  raw/ioat: add start and stop functions for idxd devices
  raw/ioat: add data path for idxd devices
  raw/ioat: add info function for idxd devices
  raw/ioat: create separate statistics structure
  raw/ioat: move xstats functions to common file
  raw/ioat: add xstats tracking for idxd devices
  raw/ioat: clean up use of common test function

Cheng Jiang (1):
  raw/ioat: add a flag to control copying handle parameters

Kevin Laatz (3):
  usertools/dpdk-devbind.py: add support for DSA HW
  raw/ioat: add vdev probe for DSA/idxd devices
  raw/ioat: create rawdev instances for idxd vdevs

 doc/guides/rawdevs/ioat.rst   | 131 +++--
 drivers/raw/ioat/dpdk_idxd_cfg.py |  79 +++
 drivers/raw/ioat/idxd_pci.c   | 344 +
 drivers/raw/ioat/idxd_vdev.c  | 234 +
 drivers/raw/ioat/ioat_common.c| 237 +
 drivers/raw/ioat/ioat_private.h   |  80 +++
 drivers/raw/ioat/ioat_rawdev.c|  95 +---
 drivers/raw/ioat/ioat_rawdev_test.c   |   1 +
 .../raw/ioat/{rte_ioat_spec.h => ioat_spec.h} |  90 +++-
 drivers/raw/ioat/meson.build  |  15 +-
 drivers/raw/ioat/rte_ioat_rawdev.h| 169 ++-
 drivers/raw/ioat/rte_ioat_rawdev_fns.h| 474 ++
 usertools/dpdk-devbind.py |   4 +-
 13 files changed, 1646 insertions(+), 307 deletions(-)
 create mode 100755 drivers/raw/ioat/dpdk_idxd_cfg.py
 create mode 100644 drivers/raw/ioat/idxd_pci.c
 create mode 100644 drivers/raw/ioat/idxd_vdev.c
 create mode 100644 drivers/raw/ioat/ioat_common.c
 create mode 100644 drivers/raw/ioat/ioat_private.h
 rename drivers/raw/ioat/{rte_ioat_spec.h => ioat_spec.h} (74%)
 create mode 100644 drivers/raw/ioat/rte_ioat_rawdev_fns.h

-- 
2.25.1

[dpdk-dev] [PATCH v2 03/18] raw/ioat: make the HW register spec private

[Bruce Richardson]

Only a few definitions from the hardware spec are actually used in the
driver runtime, so we can copy over those few and make the rest of the spec
a private header in the driver.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/ioat_rawdev.c|  3 ++
 .../raw/ioat/{rte_ioat_spec.h => ioat_spec.h} | 26 ---
 drivers/raw/ioat/meson.build  |  3 +-
 drivers/raw/ioat/rte_ioat_rawdev_fns.h| 44 +--
 4 files changed, 44 insertions(+), 32 deletions(-)
 rename drivers/raw/ioat/{rte_ioat_spec.h => ioat_spec.h} (91%)

diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c
index 856b55cb6e..bba072f2a7 100644
--- a/drivers/raw/ioat/ioat_rawdev.c
+++ b/drivers/raw/ioat/ioat_rawdev.c
@@ -4,10 +4,12 @@
 
 #include 
 #include 
+#include 
 #include 
 #include 
 
 #include "rte_ioat_rawdev.h"
+#include "ioat_spec.h"
 
 static struct rte_pci_driver ioat_pmd_drv;
 
@@ -261,6 +263,7 @@ ioat_rawdev_create(const char *name, struct rte_pci_device 
*dev)
ioat->rawdev = rawdev;
ioat->mz = mz;
ioat->regs = dev->mem_resource[0].addr;
+   ioat->doorbell = &ioat->regs->dmacount;
ioat->ring_size = 0;
ioat->desc_ring = NULL;
ioat->status_addr = ioat->mz->iova +
diff --git a/drivers/raw/ioat/rte_ioat_spec.h b/drivers/raw/ioat/ioat_spec.h
similarity index 91%
rename from drivers/raw/ioat/rte_ioat_spec.h
rename to drivers/raw/ioat/ioat_spec.h
index c6e7929b2c..9645e16d41 100644
--- a/drivers/raw/ioat/rte_ioat_spec.h
+++ b/drivers/raw/ioat/ioat_spec.h
@@ -86,32 +86,6 @@ struct rte_ioat_registers {
 
 #define RTE_IOAT_CHANCMP_ALIGN 8   /* CHANCMP address must 
be 64-bit aligned */
 
-struct rte_ioat_generic_hw_desc {
-   uint32_t size;
-   union {
-   uint32_t control_raw;
-   struct {
-   uint32_t int_enable: 1;
-   uint32_t src_snoop_disable: 1;
-   uint32_t dest_snoop_disable: 1;
-   uint32_t completion_update: 1;
-   uint32_t fence: 1;
-   uint32_t reserved2: 1;
-   uint32_t src_page_break: 1;
-   uint32_t dest_page_break: 1;
-   uint32_t bundle: 1;
-   uint32_t dest_dca: 1;
-   uint32_t hint: 1;
-   uint32_t reserved: 13;
-   uint32_t op: 8;
-   } control;
-   } u;
-   uint64_t src_addr;
-   uint64_t dest_addr;
-   uint64_t next;
-   uint64_t op_specific[4];
-};
-
 struct rte_ioat_dma_hw_desc {
uint32_t size;
union {
diff --git a/drivers/raw/ioat/meson.build b/drivers/raw/ioat/meson.build
index f66e9b605e..06636f8a9f 100644
--- a/drivers/raw/ioat/meson.build
+++ b/drivers/raw/ioat/meson.build
@@ -8,5 +8,4 @@ sources = files('ioat_rawdev.c',
 deps += ['rawdev', 'bus_pci', 'mbuf']
 
 install_headers('rte_ioat_rawdev.h',
-   'rte_ioat_rawdev_fns.h',
-   'rte_ioat_spec.h')
+   'rte_ioat_rawdev_fns.h')
diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h 
b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
index 06b4edcbb0..0cee6b1b09 100644
--- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h
+++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
@@ -5,9 +5,37 @@
 #define _RTE_IOAT_RAWDEV_FNS_H_
 
 #include 
-#include 
 #include 
-#include "rte_ioat_spec.h"
+
+/**
+ * @internal
+ * Structure representing a device descriptor
+ */
+struct rte_ioat_generic_hw_desc {
+   uint32_t size;
+   union {
+   uint32_t control_raw;
+   struct {
+   uint32_t int_enable: 1;
+   uint32_t src_snoop_disable: 1;
+   uint32_t dest_snoop_disable: 1;
+   uint32_t completion_update: 1;
+   uint32_t fence: 1;
+   uint32_t reserved2: 1;
+   uint32_t src_page_break: 1;
+   uint32_t dest_page_break: 1;
+   uint32_t bundle: 1;
+   uint32_t dest_dca: 1;
+   uint32_t hint: 1;
+   uint32_t reserved: 13;
+   uint32_t op: 8;
+   } control;
+   } u;
+   uint64_t src_addr;
+   uint64_t dest_addr;
+   uint64_t next;
+   uint64_t op_specific[4];
+};
 
 /**
  * @internal
@@ -18,7 +46,7 @@ struct rte_ioat_rawdev {
const struct rte_memzone *mz;
const struct rte_memzone *desc_mz;
 
-   volatile struct rte_ioat_registers *regs;
+   volatile uint16_t *doorbell;
phys_addr_t status_addr;
phys_addr_t ring_addr;
 
@@ -39,8 +67,16 @@ struct rte_ioat_rawdev {
 
/* to report completions, the device will write status back here */
volatile uint64_t status __rte_cache_aligned;
+
+   /* pointer to the register bar */
+   

[dpdk-dev] [PATCH v2 06/18] raw/ioat: add vdev probe for DSA/idxd devices

[Bruce Richardson]

From: Kevin Laatz 

The Intel DSA devices can be exposed to userspace via kernel driver, so can
be used without having to bind them to vfio/uio. Therefore we add support
for using those kernel-configured devices as vdevs, taking as parameter the
individual HW work queue to be used by the vdev.

Signed-off-by: Kevin Laatz 
Signed-off-by: Bruce Richardson 
---
 doc/guides/rawdevs/ioat.rst  |  68 +--
 drivers/raw/ioat/idxd_vdev.c | 123 +++
 drivers/raw/ioat/meson.build |   6 +-
 3 files changed, 192 insertions(+), 5 deletions(-)
 create mode 100644 drivers/raw/ioat/idxd_vdev.c

diff --git a/doc/guides/rawdevs/ioat.rst b/doc/guides/rawdevs/ioat.rst
index b83cf0f7db..43a69ec4c6 100644
--- a/doc/guides/rawdevs/ioat.rst
+++ b/doc/guides/rawdevs/ioat.rst
@@ -37,9 +37,62 @@ No additional compilation steps are necessary.
 Device Setup
 -
 
+Depending on support provided by the PMD, HW devices can either use the kernel 
configured driver
+or be bound to a user-space IO driver for use.
+For example, Intel\ |reg| DSA devices can use the IDXD kernel driver or 
DPDK-supported drivers,
+such as ``vfio-pci``.
+
+Intel\ |reg| DSA devices using idxd kernel driver
+~~
+
+To use a Intel\ |reg| DSA device bound to the IDXD kernel driver, the device 
must first be configured.
+The `accel-config `_ utility library can 
be used for configuration.
+
+.. note::
+The device configuration can also be done by directly interacting with 
the sysfs nodes.
+
+There are some mandatory configuration steps before being able to use a device 
with an application.
+The internal engines, which do the copies or other operations,
+and the work-queues, which are used by applications to assign work to the 
device,
+need to be assigned to groups, and the various other configuration options,
+such as priority or queue depth, need to be set for each queue.
+
+To assign an engine to a group::
+
+$ accel-config config-engine dsa0/engine0.0 --group-id=0
+$ accel-config config-engine dsa0/engine0.1 --group-id=1
+
+To assign work queues to groups for passing descriptors to the engines a 
similar accel-config command can be used.
+However, the work queues also need to be configured depending on the use-case.
+Some configuration options include:
+
+* mode (Dedicated/Shared): Indicates whether a WQ may accept jobs from 
multiple queues simultaneously.
+* priority: WQ priority between 1 and 15. Larger value means higher priority.
+* wq-size: the size of the WQ. Sum of all WQ sizes must be less that the 
total-size defined by the device.
+* type: WQ type (kernel/mdev/user). Determines how the device is presented.
+* name: identifier given to the WQ.
+
+Example configuration for a work queue::
+
+$ accel-config config-wq dsa0/wq0.0 --group-id=0 \
+   --mode=dedicated --priority=10 --wq-size=8 \
+   --type=user --name=app1
+
+Once the devices have been configured, they need to be enabled::
+
+$ accel-config enable-device dsa0
+$ accel-config enable-wq dsa0/wq0.0
+
+Check the device configuration::
+
+$ accel-config list
+
+Devices using VFIO/UIO drivers
+~~~
+
 The HW devices to be used will need to be bound to a user-space IO driver for 
use.
 The ``dpdk-devbind.py`` script can be used to view the state of the devices
-and to bind them to a suitable DPDK-supported kernel driver, such as 
``vfio-pci``.
+and to bind them to a suitable DPDK-supported driver, such as ``vfio-pci``.
 For example::
 
$ dpdk-devbind.py -b vfio-pci 00:04.0 00:04.1
@@ -47,9 +100,16 @@ For example::
 Device Probing and Initialization
 ~~
 
-Once bound to a suitable kernel device driver, the HW devices will be found
-as part of the PCI scan done at application initialization time. No vdev
-parameters need to be passed to create or initialize the device.
+For devices bound to a suitable DPDK-supported VFIO/UIO driver, the HW devices 
will
+be found as part of the device scan done at application initialization time 
without
+the need to pass parameters to the application.
+
+If the device is bound to the IDXD kernel driver (and previously configured 
with sysfs),
+then a specific work queue needs to be passed to the application via a vdev 
parameter.
+This vdev parameter take the driver name and work queue name as parameters.
+For example, to use work queue 0 on Intel\ |reg| DSA instance 0::
+
+$ dpdk-test --no-pci --vdev=rawdev_idxd,wq=0.0
 
 Once probed successfully, the device will appear as a ``rawdev``, that is a
 "raw device type" inside DPDK, and can be accessed using APIs from the
diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c
new file mode 100644
index 00..0509fc0842
--- /dev/null
+++ b/drivers/raw/ioat/idxd_vdev.c
@@ -0,0 +1,123 @@
+/* SPDX-License-Identifier: BS

[dpdk-dev] [PATCH v2 05/18] raw/ioat: add skeleton for VFIO/UIO based DSA device

[Bruce Richardson]

Add in the basic probe/remove skeleton code for DSA devices which are bound
directly to vfio or uio driver. The kernel module for supporting these uses
the "idxd" name, so that name is used as function and file prefix to avoid
conflict with existing "ioat" prefixed functions.

Since we are adding new files to the driver and there will be common
definitions shared between the various files, we create a new internal
header file ioat_private.h to hold common macros and function prototypes.

Signed-off-by: Bruce Richardson 
---
 doc/guides/rawdevs/ioat.rst | 69 ++---
 drivers/raw/ioat/idxd_pci.c | 56 ++
 drivers/raw/ioat/ioat_private.h | 27 +
 drivers/raw/ioat/ioat_rawdev.c  |  9 +
 drivers/raw/ioat/meson.build|  6 ++-
 5 files changed, 108 insertions(+), 59 deletions(-)
 create mode 100644 drivers/raw/ioat/idxd_pci.c
 create mode 100644 drivers/raw/ioat/ioat_private.h

diff --git a/doc/guides/rawdevs/ioat.rst b/doc/guides/rawdevs/ioat.rst
index c46460ff45..b83cf0f7db 100644
--- a/doc/guides/rawdevs/ioat.rst
+++ b/doc/guides/rawdevs/ioat.rst
@@ -3,10 +3,12 @@
 
 .. include:: 
 
-IOAT Rawdev Driver for Intel\ |reg| QuickData Technology
-==
+IOAT Rawdev Driver
+===
 
 The ``ioat`` rawdev driver provides a poll-mode driver (PMD) for Intel\ |reg|
+Data Streaming Accelerator `(Intel DSA)
+`_ and 
for Intel\ |reg|
 QuickData Technology, part of Intel\ |reg| I/O Acceleration Technology
 `(Intel I/OAT)
 
`_.
@@ -17,61 +19,30 @@ be done by software, freeing up CPU cycles for other tasks.
 Hardware Requirements
 --
 
-On Linux, the presence of an Intel\ |reg| QuickData Technology hardware can
-be detected by checking the output of the ``lspci`` command, where the
-hardware will be often listed as "Crystal Beach DMA" or "CBDMA". For
-example, on a system with Intel\ |reg| Xeon\ |reg| CPU E5-2699 v4 @2.20GHz,
-lspci shows:
-
-.. code-block:: console
-
-  # lspci | grep DMA
-  00:04.0 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 0 (rev 01)
-  00:04.1 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 1 (rev 01)
-  00:04.2 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 2 (rev 01)
-  00:04.3 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 3 (rev 01)
-  00:04.4 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 4 (rev 01)
-  00:04.5 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 5 (rev 01)
-  00:04.6 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 6 (rev 01)
-  00:04.7 System peripheral: Intel Corporation Xeon E7 v4/Xeon E5 v4/Xeon E3 
v4/Xeon D Crystal Beach DMA Channel 7 (rev 01)
-
-On a system with Intel\ |reg| Xeon\ |reg| Gold 6154 CPU @ 3.00GHz, lspci
-shows:
-
-.. code-block:: console
-
-  # lspci | grep DMA
-  00:04.0 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-  00:04.1 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-  00:04.2 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-  00:04.3 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-  00:04.4 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-  00:04.5 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-  00:04.6 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-  00:04.7 System peripheral: Intel Corporation Sky Lake-E CBDMA Registers (rev 
04)
-
+The ``dpdk-devbind.py`` script, included with DPDK,
+can be used to show the presence of supported hardware.
+Running ``dpdk-devbind.py --status-dev misc`` will show all the miscellaneous,
+or rawdev-based devices on the system.
+For Intel\ |reg| QuickData Technology devices, the hardware will be often 
listed as "Crystal Beach DMA",
+or "CBDMA".
+For Intel\ |reg| DSA devices, they are currently (at time of writing) 
appearing as devices with type "0b25",
+due to the absence of pci-id database entries for them at this point.
 
 Compilation
 
 
-For builds done with ``make``, the driver compilation is enabled by the
-``CONFIG_RTE_LIBRTE_PMD_IOAT_RAWDEV`` build configuration option. This is
-enabled by default in builds for x86 platforms, and disabled in other
-configurations.
-
-For builds using ``meson`` and ``ninja``, the driver will be built when the
-target platform is x86-based.
+For builds using ``meson`` and ``ninja

[dpdk-dev] [PATCH v2 04/18] usertools/dpdk-devbind.py: add support for DSA HW

[Bruce Richardson]

From: Kevin Laatz 

Intel Data Streaming Accelerator (Intel DSA) is a high-performance data
copy and transformation accelerator which will be integrated in future
Intel processors [1].

Add DSA device support to dpdk-devbind.py script.

[1] https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator

Signed-off-by: Bruce Richardson 
Signed-off-by: Kevin Laatz 
---
 usertools/dpdk-devbind.py | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/usertools/dpdk-devbind.py b/usertools/dpdk-devbind.py
index 86b6b53c40..35d4384041 100755
--- a/usertools/dpdk-devbind.py
+++ b/usertools/dpdk-devbind.py
@@ -51,6 +51,8 @@
   'SVendor': None, 'SDevice': None}
 intel_ioat_icx = {'Class': '08', 'Vendor': '8086', 'Device': '0b00',
   'SVendor': None, 'SDevice': None}
+intel_idxd_spr = {'Class': '08', 'Vendor': '8086', 'Device': '0b25',
+  'SVendor': None, 'SDevice': None}
 intel_ntb_skx = {'Class': '06', 'Vendor': '8086', 'Device': '201c',
   'SVendor': None, 'SDevice': None}
 
@@ -60,7 +62,7 @@
 eventdev_devices = [cavium_sso, cavium_tim, octeontx2_sso]
 mempool_devices = [cavium_fpa, octeontx2_npa]
 compress_devices = [cavium_zip]
-misc_devices = [intel_ioat_bdw, intel_ioat_skx, intel_ioat_icx, intel_ntb_skx, 
octeontx2_dma]
+misc_devices = [intel_ioat_bdw, intel_ioat_skx, intel_ioat_icx, 
intel_idxd_spr, intel_ntb_skx, octeontx2_dma]
 
 # global dict ethernet devices present. Dictionary indexed by PCI address.
 # Each device within this is itself a dictionary of device properties
-- 
2.25.1

[dpdk-dev] [PATCH v2 08/18] raw/ioat: create rawdev instances on idxd PCI probe

[Bruce Richardson]

When a matching device is found via PCI probe create a rawdev instance for
each queue on the hardware. Use empty self-test function for these devices
so that the overall rawdev_autotest does not report failures.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_pci.c| 236 -
 drivers/raw/ioat/ioat_common.c |  61 +++
 drivers/raw/ioat/ioat_private.h|  31 
 drivers/raw/ioat/ioat_rawdev_test.c|   7 +
 drivers/raw/ioat/ioat_spec.h   |  64 +++
 drivers/raw/ioat/meson.build   |   1 +
 drivers/raw/ioat/rte_ioat_rawdev_fns.h |  35 +++-
 7 files changed, 432 insertions(+), 3 deletions(-)
 create mode 100644 drivers/raw/ioat/ioat_common.c

diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c
index 1a30e9c316..72f4ecebb7 100644
--- a/drivers/raw/ioat/idxd_pci.c
+++ b/drivers/raw/ioat/idxd_pci.c
@@ -3,8 +3,10 @@
  */
 
 #include 
+#include 
 
 #include "ioat_private.h"
+#include "ioat_spec.h"
 
 #define IDXD_VENDOR_ID 0x8086
 #define IDXD_DEVICE_ID_SPR 0x0B25
@@ -16,17 +18,245 @@ const struct rte_pci_id pci_id_idxd_map[] = {
{ .vendor_id = 0, /* sentinel */ },
 };
 
+static inline int
+idxd_pci_dev_command(struct idxd_rawdev *idxd, enum rte_idxd_cmds command)
+{
+   uint8_t err_code;
+   uint16_t qid = idxd->qid;
+   int i = 0;
+
+   if (command >= idxd_disable_wq && command <= idxd_reset_wq)
+   qid = (1 << qid);
+   rte_spinlock_lock(&idxd->u.pci->lk);
+   idxd->u.pci->regs->cmd = (command << IDXD_CMD_SHIFT) | qid;
+
+   do {
+   rte_pause();
+   err_code = idxd->u.pci->regs->cmdstatus;
+   if (++i >= 1000) {
+   IOAT_PMD_ERR("Timeout waiting for command response from 
HW");
+   rte_spinlock_unlock(&idxd->u.pci->lk);
+   return err_code;
+   }
+   } while (idxd->u.pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK);
+   rte_spinlock_unlock(&idxd->u.pci->lk);
+
+   return err_code & CMDSTATUS_ERR_MASK;
+}
+
+static int
+idxd_is_wq_enabled(struct idxd_rawdev *idxd)
+{
+   uint32_t state = idxd->u.pci->wq_regs[idxd->qid].wqcfg[WQ_STATE_IDX];
+   return ((state >> WQ_STATE_SHIFT) & WQ_STATE_MASK) == 0x1;
+}
+
+static const struct rte_rawdev_ops idxd_pci_ops = {
+   .dev_selftest = idxd_rawdev_test
+};
+
+/* each portal uses 4 x 4k pages */
+#define IDXD_PORTAL_SIZE (4096 * 4)
+
+static int
+init_pci_device(struct rte_pci_device *dev, struct idxd_rawdev *idxd)
+{
+   struct idxd_pci_common *pci;
+   uint8_t nb_groups, nb_engines, nb_wqs;
+   uint16_t grp_offset, wq_offset; /* how far into bar0 the regs are */
+   uint16_t wq_size, total_wq_size;
+   uint8_t lg2_max_batch, lg2_max_copy_size;
+   unsigned int i, err_code;
+
+   pci = malloc(sizeof(*pci));
+   if (pci == NULL) {
+   IOAT_PMD_ERR("%s: Can't allocate memory", __func__);
+   goto err;
+   }
+   rte_spinlock_init(&pci->lk);
+
+   /* assign the bar registers, and then configure device */
+   pci->regs = dev->mem_resource[0].addr;
+   grp_offset = (uint16_t)pci->regs->offsets[0];
+   pci->grp_regs = RTE_PTR_ADD(pci->regs, grp_offset * 0x100);
+   wq_offset = (uint16_t)(pci->regs->offsets[0] >> 16);
+   pci->wq_regs = RTE_PTR_ADD(pci->regs, wq_offset * 0x100);
+   pci->portals = dev->mem_resource[2].addr;
+
+   /* sanity check device status */
+   if (pci->regs->gensts & GENSTS_DEV_STATE_MASK) {
+   /* need function-level-reset (FLR) or is enabled */
+   IOAT_PMD_ERR("Device status is not disabled, cannot init");
+   goto err;
+   }
+   if (pci->regs->cmdstatus & CMDSTATUS_ACTIVE_MASK) {
+   /* command in progress */
+   IOAT_PMD_ERR("Device has a command in progress, cannot init");
+   goto err;
+   }
+
+   /* read basic info about the hardware for use when configuring */
+   nb_groups = (uint8_t)pci->regs->grpcap;
+   nb_engines = (uint8_t)pci->regs->engcap;
+   nb_wqs = (uint8_t)(pci->regs->wqcap >> 16);
+   total_wq_size = (uint16_t)pci->regs->wqcap;
+   lg2_max_copy_size = (uint8_t)(pci->regs->gencap >> 16) & 0x1F;
+   lg2_max_batch = (uint8_t)(pci->regs->gencap >> 21) & 0x0F;
+
+   IOAT_PMD_DEBUG("nb_groups = %u, nb_engines = %u, nb_wqs = %u",
+   nb_groups, nb_engines, nb_wqs);
+
+   /* zero out any old config */
+   for (i = 0; i < nb_groups; i++) {
+   pci->grp_regs[i].grpengcfg = 0;
+   pci->grp_regs[i].grpwqcfg[0] = 0;
+   }
+   for (i = 0; i < nb_wqs; i++)
+   pci->wq_regs[i].wqcfg[0] = 0;
+
+   /* put each engine into a separate group to avoid reordering */
+   if (nb_groups > nb_engines)
+   nb_groups = nb_engines;
+   if (nb_groups < nb_engine

[dpdk-dev] [PATCH v2 09/18] raw/ioat: create rawdev instances for idxd vdevs

[Bruce Richardson]

From: Kevin Laatz 

For each vdev (DSA work queue) instance, create a rawdev instance.

Signed-off-by: Kevin Laatz 
Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_vdev.c| 107 +++-
 drivers/raw/ioat/ioat_private.h |   4 ++
 2 files changed, 110 insertions(+), 1 deletion(-)

diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c
index 0509fc0842..93c023a6e8 100644
--- a/drivers/raw/ioat/idxd_vdev.c
+++ b/drivers/raw/ioat/idxd_vdev.c
@@ -2,6 +2,12 @@
  * Copyright(c) 2020 Intel Corporation
  */
 
+#include 
+#include 
+#include 
+#include 
+
+#include 
 #include 
 #include 
 #include 
@@ -24,6 +30,35 @@ struct idxd_vdev_args {
uint8_t wq_id;
 };
 
+static const struct rte_rawdev_ops idxd_vdev_ops = {
+   .dev_selftest = idxd_rawdev_test,
+};
+
+static void *
+idxd_vdev_mmap_wq(struct idxd_vdev_args *args)
+{
+   void *addr;
+   char path[PATH_MAX];
+   int fd;
+
+   snprintf(path, sizeof(path), "/dev/dsa/wq%u.%u",
+   args->device_id, args->wq_id);
+   fd = open(path, O_RDWR);
+   if (fd < 0) {
+   IOAT_PMD_ERR("Failed to open device path");
+   return NULL;
+   }
+
+   addr = mmap(NULL, 0x1000, PROT_WRITE, MAP_SHARED | MAP_POPULATE, fd, 0);
+   close(fd);
+   if (addr == MAP_FAILED) {
+   IOAT_PMD_ERR("Failed to mmap device");
+   return NULL;
+   }
+
+   return addr;
+}
+
 static int
 idxd_rawdev_parse_wq(const char *key __rte_unused, const char *value,
  void *extra_args)
@@ -70,10 +105,34 @@ idxd_vdev_parse_params(struct rte_kvargs *kvlist, struct 
idxd_vdev_args *args)
return -EINVAL;
 }
 
+static int
+idxd_vdev_get_max_batches(struct idxd_vdev_args *args)
+{
+   char sysfs_path[PATH_MAX];
+   FILE *f;
+   int ret = -1;
+
+   snprintf(sysfs_path, sizeof(sysfs_path),
+   "/sys/bus/dsa/devices/wq%u.%u/size",
+   args->device_id, args->wq_id);
+   f = fopen(sysfs_path, "r");
+   if (f == NULL)
+   return -1;
+
+   fscanf(f, "%d", &ret);
+   /* if fscanf does not read ret correctly, it will be -1, so no error
+* handling is necessary
+*/
+
+   fclose(f);
+   return ret;
+}
+
 static int
 idxd_rawdev_probe_vdev(struct rte_vdev_device *vdev)
 {
struct rte_kvargs *kvlist;
+   struct idxd_rawdev idxd = {0};
struct idxd_vdev_args vdev_args;
const char *name;
int ret = 0;
@@ -96,13 +155,32 @@ idxd_rawdev_probe_vdev(struct rte_vdev_device *vdev)
return -EINVAL;
}
 
+   idxd.qid = vdev_args.wq_id;
+   idxd.u.vdev.dsa_id = vdev_args.device_id;
+   idxd.max_batches = idxd_vdev_get_max_batches(&vdev_args);
+
+   idxd.public.portal = idxd_vdev_mmap_wq(&vdev_args);
+   if (idxd.public.portal == NULL) {
+   IOAT_PMD_ERR("WQ mmap failed");
+   return -ENOENT;
+   }
+
+   ret = idxd_rawdev_create(name, &vdev->device, &idxd, &idxd_vdev_ops);
+   if (ret) {
+   IOAT_PMD_ERR("Failed to create rawdev %s", name);
+   return ret;
+   }
+
return 0;
 }
 
 static int
 idxd_rawdev_remove_vdev(struct rte_vdev_device *vdev)
 {
+   struct idxd_rawdev *idxd;
const char *name;
+   struct rte_rawdev *rdev;
+   int ret = 0;
 
name = rte_vdev_device_name(vdev);
if (name == NULL)
@@ -110,7 +188,34 @@ idxd_rawdev_remove_vdev(struct rte_vdev_device *vdev)
 
IOAT_PMD_INFO("Remove DSA vdev %p", name);
 
-   return 0;
+   rdev = rte_rawdev_pmd_get_named_dev(name);
+   if (!rdev) {
+   IOAT_PMD_ERR("Invalid device name (%s)", name);
+   return -EINVAL;
+   }
+
+   idxd = rdev->dev_private;
+
+   /* free context and memory */
+   if (rdev->dev_private != NULL) {
+   IOAT_PMD_DEBUG("Freeing device driver memory");
+   rdev->dev_private = NULL;
+
+   if (munmap(idxd->public.portal, 0x1000) < 0) {
+   IOAT_PMD_ERR("Error unmapping portal");
+   ret = -errno;
+   }
+
+   rte_free(idxd->public.batch_ring);
+   rte_free(idxd->public.hdl_ring);
+
+   rte_memzone_free(idxd->mz);
+   }
+
+   if (rte_rawdev_pmd_release(rdev))
+   IOAT_PMD_ERR("Device cleanup failed");
+
+   return ret;
 }
 
 struct rte_vdev_driver idxd_rawdev_drv_vdev = {
diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h
index 32c824536d..bd2bef3834 100644
--- a/drivers/raw/ioat/ioat_private.h
+++ b/drivers/raw/ioat/ioat_private.h
@@ -45,6 +45,10 @@ struct idxd_rawdev {
uint16_t max_batches;
 
union {
+   struct {
+   unsigned dsa_id;
+   } vdev;
+
struct idxd_pci_comm

[dpdk-dev] [PATCH v2 07/18] raw/ioat: include example configuration script

[Bruce Richardson]

Devices managed by the idxd kernel driver must be configured for DPDK use
before it can be used by the ioat driver. This example script serves both
as a quick way to get the driver set up with a simple configuration, and as
the basis for users to modify it and create their own configuration
scripts.

Signed-off-by: Bruce Richardson 
---
 doc/guides/rawdevs/ioat.rst   |  2 +
 drivers/raw/ioat/dpdk_idxd_cfg.py | 79 +++
 2 files changed, 81 insertions(+)
 create mode 100755 drivers/raw/ioat/dpdk_idxd_cfg.py

diff --git a/doc/guides/rawdevs/ioat.rst b/doc/guides/rawdevs/ioat.rst
index 43a69ec4c6..8d241d7e77 100644
--- a/doc/guides/rawdevs/ioat.rst
+++ b/doc/guides/rawdevs/ioat.rst
@@ -50,6 +50,8 @@ The `accel-config `_ 
utility library can b
 
 .. note::
 The device configuration can also be done by directly interacting with 
the sysfs nodes.
+An example of how this may be done can be seen in the script 
``dpdk_idxd_cfg.py``
+included in the driver source directory.
 
 There are some mandatory configuration steps before being able to use a device 
with an application.
 The internal engines, which do the copies or other operations,
diff --git a/drivers/raw/ioat/dpdk_idxd_cfg.py 
b/drivers/raw/ioat/dpdk_idxd_cfg.py
new file mode 100755
index 00..bce4bb5bd4
--- /dev/null
+++ b/drivers/raw/ioat/dpdk_idxd_cfg.py
@@ -0,0 +1,79 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2020 Intel Corporation
+
+"""
+Configure an entire Intel DSA instance, using idxd kernel driver, for DPDK use
+"""
+
+import sys
+import argparse
+import os
+import os.path
+
+
+class SysfsDir:
+"Used to read/write paths in a sysfs directory"
+def __init__(self, path):
+self.path = path
+
+def read_int(self, filename):
+"Return a value from sysfs file"
+with open(os.path.join(self.path, filename)) as f:
+return int(f.readline())
+
+def write_values(self, values):
+"write dictionary, where key is filename and value is value to write"
+for filename, contents in values.items():
+with open(os.path.join(self.path, filename), "w") as f:
+f.write(str(contents))
+
+
+def configure_dsa(dsa_id, queues):
+"Configure the DSA instance with appropriate number of queues"
+dsa_dir = SysfsDir(f"/sys/bus/dsa/devices/dsa{dsa_id}")
+drv_dir = SysfsDir("/sys/bus/dsa/drivers/dsa")
+
+max_groups = dsa_dir.read_int("max_groups")
+max_engines = dsa_dir.read_int("max_engines")
+max_queues = dsa_dir.read_int("max_work_queues")
+max_tokens = dsa_dir.read_int("max_tokens")
+
+# we want one engine per group
+nb_groups = min(max_engines, max_groups)
+for grp in range(nb_groups):
+dsa_dir.write_values({f"engine{dsa_id}.{grp}/group_id": grp})
+
+nb_queues = min(queues, max_queues)
+if queues > nb_queues:
+print(f"Setting number of queues to max supported value: {max_queues}")
+
+# configure each queue
+for q in range(nb_queues):
+wq_dir = SysfsDir(os.path.join(dsa_dir.path, f"wq{dsa_id}.{q}"))
+wq_dir.write_values({"group_id": q % nb_groups,
+ "type": "user",
+ "mode": "dedicated",
+ "name": f"dpdk_wq{dsa_id}.{q}",
+ "priority": 1,
+ "size": int(max_tokens / nb_queues)})
+
+# enable device and then queues
+drv_dir.write_values({"bind": f"dsa{dsa_id}"})
+for q in range(nb_queues):
+drv_dir.write_values({"bind": f"wq{dsa_id}.{q}"})
+
+
+def main(args):
+"Main function, does arg parsing and calls config function"
+arg_p = argparse.ArgumentParser(
+description="Configure whole DSA device instance for DPDK use")
+arg_p.add_argument('dsa_id', type=int, help="DSA instance number")
+arg_p.add_argument('-q', metavar='queues', type=int, default=255,
+   help="Number of queues to set up")
+parsed_args = arg_p.parse_args(args[1:])
+configure_dsa(parsed_args.dsa_id, parsed_args.q)
+
+
+if __name__ == "__main__":
+main(sys.argv)
-- 
2.25.1

[dpdk-dev] [PATCH v2 10/18] raw/ioat: add datapath data structures for idxd devices

[Bruce Richardson]

Add in the relevant data structures for the data path for DSA devices. Also
include a device dump function to output the status of each device.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_pci.c|  3 +-
 drivers/raw/ioat/idxd_vdev.c   |  1 +
 drivers/raw/ioat/ioat_common.c | 34 +++
 drivers/raw/ioat/ioat_private.h|  2 +
 drivers/raw/ioat/ioat_rawdev_test.c|  3 +-
 drivers/raw/ioat/rte_ioat_rawdev_fns.h | 80 ++
 6 files changed, 121 insertions(+), 2 deletions(-)

diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c
index 72f4ecebb7..ce238ae04c 100644
--- a/drivers/raw/ioat/idxd_pci.c
+++ b/drivers/raw/ioat/idxd_pci.c
@@ -52,7 +52,8 @@ idxd_is_wq_enabled(struct idxd_rawdev *idxd)
 }
 
 static const struct rte_rawdev_ops idxd_pci_ops = {
-   .dev_selftest = idxd_rawdev_test
+   .dev_selftest = idxd_rawdev_test,
+   .dump = idxd_dev_dump,
 };
 
 /* each portal uses 4 x 4k pages */
diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c
index 93c023a6e8..0f9aa48e84 100644
--- a/drivers/raw/ioat/idxd_vdev.c
+++ b/drivers/raw/ioat/idxd_vdev.c
@@ -32,6 +32,7 @@ struct idxd_vdev_args {
 
 static const struct rte_rawdev_ops idxd_vdev_ops = {
.dev_selftest = idxd_rawdev_test,
+   .dump = idxd_dev_dump,
 };
 
 static void *
diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c
index 37a14e514d..fb4f7055de 100644
--- a/drivers/raw/ioat/ioat_common.c
+++ b/drivers/raw/ioat/ioat_common.c
@@ -7,6 +7,36 @@
 
 #include "ioat_private.h"
 
+int
+idxd_dev_dump(struct rte_rawdev *dev, FILE *f)
+{
+   struct idxd_rawdev *idxd = dev->dev_private;
+   struct rte_idxd_rawdev *rte_idxd = &idxd->public;
+   int i;
+
+   fprintf(f, "Raw Device #%d\n", dev->dev_id);
+   fprintf(f, "Driver: %s\n\n", dev->driver_name);
+
+   fprintf(f, "Portal: %p\n", rte_idxd->portal);
+   fprintf(f, "Batch Ring size: %u\n", rte_idxd->batch_ring_sz);
+   fprintf(f, "Comp Handle Ring size: %u\n\n", rte_idxd->hdl_ring_sz);
+
+   fprintf(f, "Next batch: %u\n", rte_idxd->next_batch);
+   fprintf(f, "Next batch to be completed: %u\n", 
rte_idxd->next_completed);
+   for (i = 0; i < rte_idxd->batch_ring_sz; i++) {
+   struct rte_idxd_desc_batch *b = &rte_idxd->batch_ring[i];
+   fprintf(f, "Batch %u @%p: submitted=%u, op_count=%u, 
hdl_end=%u\n",
+   i, b, b->submitted, b->op_count, b->hdl_end);
+   }
+
+   fprintf(f, "\n");
+   fprintf(f, "Next free hdl: %u\n", rte_idxd->next_free_hdl);
+   fprintf(f, "Last completed hdl: %u\n", rte_idxd->last_completed_hdl);
+   fprintf(f, "Next returned hdl: %u\n", rte_idxd->next_ret_hdl);
+
+   return 0;
+}
+
 int
 idxd_rawdev_create(const char *name, struct rte_device *dev,
   const struct idxd_rawdev *base_idxd,
@@ -18,6 +48,10 @@ idxd_rawdev_create(const char *name, struct rte_device *dev,
char mz_name[RTE_MEMZONE_NAMESIZE];
int ret = 0;
 
+   RTE_BUILD_BUG_ON(sizeof(struct rte_idxd_hw_desc) != 64);
+   RTE_BUILD_BUG_ON(offsetof(struct rte_idxd_hw_desc, size) != 32);
+   RTE_BUILD_BUG_ON(sizeof(struct rte_idxd_completion) != 32);
+
if (!name) {
IOAT_PMD_ERR("Invalid name of the device!");
ret = -EINVAL;
diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h
index bd2bef3834..974eeb0106 100644
--- a/drivers/raw/ioat/ioat_private.h
+++ b/drivers/raw/ioat/ioat_private.h
@@ -59,4 +59,6 @@ extern int idxd_rawdev_create(const char *name, struct 
rte_device *dev,
 
 extern int idxd_rawdev_test(uint16_t dev_id);
 
+extern int idxd_dev_dump(struct rte_rawdev *dev, FILE *f);
+
 #endif /* _IOAT_PRIVATE_H_ */
diff --git a/drivers/raw/ioat/ioat_rawdev_test.c 
b/drivers/raw/ioat/ioat_rawdev_test.c
index 4f14b3d1bd..082b3091c4 100644
--- a/drivers/raw/ioat/ioat_rawdev_test.c
+++ b/drivers/raw/ioat/ioat_rawdev_test.c
@@ -271,7 +271,8 @@ ioat_rawdev_test(uint16_t dev_id)
 }
 
 int
-idxd_rawdev_test(uint16_t dev_id __rte_unused)
+idxd_rawdev_test(uint16_t dev_id)
 {
+   rte_rawdev_dump(dev_id, stdout);
return 0;
 }
diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h 
b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
index db377fbfa7..d258ad9fd2 100644
--- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h
+++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
@@ -88,6 +88,86 @@ struct rte_ioat_rawdev {
 #define RTE_IOAT_CHANSTS_HALTED0x3
 #define RTE_IOAT_CHANSTS_ARMED 0x4
 
+/*
+ * Defines used in the data path for interacting with hardware.
+ */
+#define IDXD_CMD_OP_SHIFT 24
+enum rte_idxd_ops {
+   idxd_op_nop = 0,
+   idxd_op_batch,
+   idxd_op_drain,
+   idxd_op_memmove,
+   idxd_op_fill
+};
+
+#define IDXD_FLAG_FENCE (1 << 0)
+#define IDXD_FLAG_COMPLET

[dpdk-dev] [PATCH v2 11/18] raw/ioat: add configure function for idxd devices

[Bruce Richardson]

Add configure function for idxd devices, taking the same parameters as the
existing configure function for ioat. The ring_size parameter is used to
compute the maximum number of bursts to be supported by the driver, given
that the hardware works on individual bursts of descriptors at a time.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_pci.c|  1 +
 drivers/raw/ioat/idxd_vdev.c   |  1 +
 drivers/raw/ioat/ioat_common.c | 64 ++
 drivers/raw/ioat/ioat_private.h|  3 ++
 drivers/raw/ioat/rte_ioat_rawdev_fns.h |  1 +
 5 files changed, 70 insertions(+)

diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c
index ce238ae04c..98e8668e34 100644
--- a/drivers/raw/ioat/idxd_pci.c
+++ b/drivers/raw/ioat/idxd_pci.c
@@ -54,6 +54,7 @@ idxd_is_wq_enabled(struct idxd_rawdev *idxd)
 static const struct rte_rawdev_ops idxd_pci_ops = {
.dev_selftest = idxd_rawdev_test,
.dump = idxd_dev_dump,
+   .dev_configure = idxd_dev_configure,
 };
 
 /* each portal uses 4 x 4k pages */
diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c
index 0f9aa48e84..73cb5d938f 100644
--- a/drivers/raw/ioat/idxd_vdev.c
+++ b/drivers/raw/ioat/idxd_vdev.c
@@ -33,6 +33,7 @@ struct idxd_vdev_args {
 static const struct rte_rawdev_ops idxd_vdev_ops = {
.dev_selftest = idxd_rawdev_test,
.dump = idxd_dev_dump,
+   .dev_configure = idxd_dev_configure,
 };
 
 static void *
diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c
index fb4f7055de..85b13b0bae 100644
--- a/drivers/raw/ioat/ioat_common.c
+++ b/drivers/raw/ioat/ioat_common.c
@@ -37,6 +37,70 @@ idxd_dev_dump(struct rte_rawdev *dev, FILE *f)
return 0;
 }
 
+int
+idxd_dev_configure(const struct rte_rawdev *dev,
+   rte_rawdev_obj_t config, size_t config_size)
+{
+   struct idxd_rawdev *idxd = dev->dev_private;
+   struct rte_idxd_rawdev *rte_idxd = &idxd->public;
+   struct rte_ioat_rawdev_config *cfg = config;
+   uint16_t max_desc = cfg->ring_size;
+   uint16_t max_batches = max_desc / BATCH_SIZE;
+   uint16_t i;
+
+   if (config_size != sizeof(*cfg))
+   return -EINVAL;
+
+   if (dev->started) {
+   IOAT_PMD_ERR("%s: Error, device is started.", __func__);
+   return -EAGAIN;
+   }
+
+   rte_idxd->hdls_disable = cfg->hdls_disable;
+
+   /* limit the batches to what can be stored in hardware */
+   if (max_batches > idxd->max_batches) {
+   IOAT_PMD_DEBUG("Ring size of %u is too large for this device, 
need to limit to %u batches of %u",
+   max_desc, idxd->max_batches, BATCH_SIZE);
+   max_batches = idxd->max_batches;
+   max_desc = max_batches * BATCH_SIZE;
+   }
+   if (!rte_is_power_of_2(max_desc))
+   max_desc = rte_align32pow2(max_desc);
+   IOAT_PMD_DEBUG("Rawdev %u using %u descriptors in %u batches",
+   dev->dev_id, max_desc, max_batches);
+
+   /* in case we are reconfiguring a device, free any existing memory */
+   rte_free(rte_idxd->batch_ring);
+   rte_free(rte_idxd->hdl_ring);
+
+   rte_idxd->batch_ring = rte_zmalloc(NULL,
+   sizeof(*rte_idxd->batch_ring) * max_batches, 0);
+   if (rte_idxd->batch_ring == NULL)
+   return -ENOMEM;
+
+   rte_idxd->hdl_ring = rte_zmalloc(NULL,
+   sizeof(*rte_idxd->hdl_ring) * max_desc, 0);
+   if (rte_idxd->hdl_ring == NULL) {
+   rte_free(rte_idxd->batch_ring);
+   rte_idxd->batch_ring = NULL;
+   return -ENOMEM;
+   }
+   rte_idxd->batch_ring_sz = max_batches;
+   rte_idxd->hdl_ring_sz = max_desc;
+
+   for (i = 0; i < rte_idxd->batch_ring_sz; i++) {
+   struct rte_idxd_desc_batch *b = &rte_idxd->batch_ring[i];
+   b->batch_desc.completion = rte_mem_virt2iova(&b->comp);
+   b->batch_desc.desc_addr = rte_mem_virt2iova(&b->null_desc);
+   b->batch_desc.op_flags = (idxd_op_batch << IDXD_CMD_OP_SHIFT) |
+   IDXD_FLAG_COMPLETION_ADDR_VALID |
+   IDXD_FLAG_REQUEST_COMPLETION;
+   }
+
+   return 0;
+}
+
 int
 idxd_rawdev_create(const char *name, struct rte_device *dev,
   const struct idxd_rawdev *base_idxd,
diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h
index 974eeb0106..928c9b497c 100644
--- a/drivers/raw/ioat/ioat_private.h
+++ b/drivers/raw/ioat/ioat_private.h
@@ -57,6 +57,9 @@ extern int idxd_rawdev_create(const char *name, struct 
rte_device *dev,
   const struct idxd_rawdev *idxd,
   const struct rte_rawdev_ops *ops);
 
+extern int idxd_dev_configure(const struct rte_rawdev *dev,
+   rte

[dpdk-dev] [PATCH v2 12/18] raw/ioat: add start and stop functions for idxd devices

[Bruce Richardson]

Add the start and stop functions for DSA hardware devices using the
vfio/uio kernel drivers. For vdevs using the idxd kernel driver, the device
must be started using sysfs before the device node appears for vdev use -
making start/stop functions in the driver unnecessary.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_pci.c | 50 +
 1 file changed, 50 insertions(+)

diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c
index 98e8668e34..13a0a03211 100644
--- a/drivers/raw/ioat/idxd_pci.c
+++ b/drivers/raw/ioat/idxd_pci.c
@@ -51,10 +51,60 @@ idxd_is_wq_enabled(struct idxd_rawdev *idxd)
return ((state >> WQ_STATE_SHIFT) & WQ_STATE_MASK) == 0x1;
 }
 
+static void
+idxd_pci_dev_stop(struct rte_rawdev *dev)
+{
+   struct idxd_rawdev *idxd = dev->dev_private;
+   uint8_t err_code;
+
+   if (!idxd_is_wq_enabled(idxd)) {
+   IOAT_PMD_ERR("Work queue %d already disabled", idxd->qid);
+   return;
+   }
+
+   err_code = idxd_pci_dev_command(idxd, idxd_disable_wq);
+   if (err_code || idxd_is_wq_enabled(idxd)) {
+   IOAT_PMD_ERR("Failed disabling work queue %d, error code: %#x",
+   idxd->qid, err_code);
+   return;
+   }
+   IOAT_PMD_DEBUG("Work queue %d disabled OK", idxd->qid);
+}
+
+static int
+idxd_pci_dev_start(struct rte_rawdev *dev)
+{
+   struct idxd_rawdev *idxd = dev->dev_private;
+   uint8_t err_code;
+
+   if (idxd_is_wq_enabled(idxd)) {
+   IOAT_PMD_WARN("WQ %d already enabled", idxd->qid);
+   return 0;
+   }
+
+   if (idxd->public.batch_ring == NULL) {
+   IOAT_PMD_ERR("WQ %d has not been fully configured", idxd->qid);
+   return -EINVAL;
+   }
+
+   err_code = idxd_pci_dev_command(idxd, idxd_enable_wq);
+   if (err_code || !idxd_is_wq_enabled(idxd)) {
+   IOAT_PMD_ERR("Failed enabling work queue %d, error code: %#x",
+   idxd->qid, err_code);
+   return err_code == 0 ? -1 : err_code;
+   }
+
+   IOAT_PMD_DEBUG("Work queue %d enabled OK", idxd->qid);
+
+   return 0;
+}
+
 static const struct rte_rawdev_ops idxd_pci_ops = {
.dev_selftest = idxd_rawdev_test,
.dump = idxd_dev_dump,
.dev_configure = idxd_dev_configure,
+   .dev_start = idxd_pci_dev_start,
+   .dev_stop = idxd_pci_dev_stop,
 };
 
 /* each portal uses 4 x 4k pages */
-- 
2.25.1

[dpdk-dev] [PATCH v2 13/18] raw/ioat: add data path for idxd devices

[Bruce Richardson]

Add support for doing copies using DSA hardware. This is implemented by
just switching on the device type field at the start of the inline
functions. Since there is no hardware which will have both device types
present this branch will always be predictable after the first call,
meaning it has little to no perf penalty.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/ioat_common.c |   1 +
 drivers/raw/ioat/ioat_rawdev.c |   1 +
 drivers/raw/ioat/rte_ioat_rawdev_fns.h | 164 +++--
 3 files changed, 157 insertions(+), 9 deletions(-)

diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c
index 85b13b0bae..23a9d71dd3 100644
--- a/drivers/raw/ioat/ioat_common.c
+++ b/drivers/raw/ioat/ioat_common.c
@@ -146,6 +146,7 @@ idxd_rawdev_create(const char *name, struct rte_device *dev,
 
idxd = rawdev->dev_private;
*idxd = *base_idxd; /* copy over the main fields already passed in */
+   idxd->public.type = RTE_IDXD_DEV;
idxd->rawdev = rawdev;
idxd->mz = mz;
 
diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c
index 20c8b671a6..15133737b9 100644
--- a/drivers/raw/ioat/ioat_rawdev.c
+++ b/drivers/raw/ioat/ioat_rawdev.c
@@ -253,6 +253,7 @@ ioat_rawdev_create(const char *name, struct rte_pci_device 
*dev)
rawdev->driver_name = dev->device.driver->name;
 
ioat = rawdev->dev_private;
+   ioat->type = RTE_IOAT_DEV;
ioat->rawdev = rawdev;
ioat->mz = mz;
ioat->regs = dev->mem_resource[0].addr;
diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h 
b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
index 1939437d50..1950c8 100644
--- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h
+++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
@@ -194,8 +194,8 @@ struct rte_idxd_rawdev {
 /**
  * Enqueue a copy operation onto the ioat device
  */
-static inline int
-rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst,
+static __rte_always_inline int
+__ioat_enqueue_copy(int dev_id, phys_addr_t src, phys_addr_t dst,
unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl,
int fence)
 {
@@ -233,8 +233,8 @@ rte_ioat_enqueue_copy(int dev_id, phys_addr_t src, 
phys_addr_t dst,
 /**
  * Trigger hardware to begin performing enqueued copy operations
  */
-static inline void
-rte_ioat_do_copies(int dev_id)
+static __rte_always_inline void
+__ioat_perform_ops(int dev_id)
 {
struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private;
ioat->desc_ring[(ioat->next_write - 1) & (ioat->ring_size - 1)].u
@@ -248,8 +248,8 @@ rte_ioat_do_copies(int dev_id)
  * @internal
  * Returns the index of the last completed operation.
  */
-static inline int
-rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error)
+static __rte_always_inline int
+__ioat_get_last_completed(struct rte_ioat_rawdev *ioat, int *error)
 {
uint64_t status = ioat->status;
 
@@ -263,8 +263,8 @@ rte_ioat_get_last_completed(struct rte_ioat_rawdev *ioat, 
int *error)
 /**
  * Returns details of copy operations that have been completed
  */
-static inline int
-rte_ioat_completed_copies(int dev_id, uint8_t max_copies,
+static __rte_always_inline int
+__ioat_completed_ops(int dev_id, uint8_t max_copies,
uintptr_t *src_hdls, uintptr_t *dst_hdls)
 {
struct rte_ioat_rawdev *ioat = rte_rawdevs[dev_id].dev_private;
@@ -274,7 +274,7 @@ rte_ioat_completed_copies(int dev_id, uint8_t max_copies,
int error;
int i = 0;
 
-   end_read = (rte_ioat_get_last_completed(ioat, &error) + 1) & mask;
+   end_read = (__ioat_get_last_completed(ioat, &error) + 1) & mask;
count = (end_read - (read & mask)) & mask;
 
if (error) {
@@ -311,4 +311,150 @@ rte_ioat_completed_copies(int dev_id, uint8_t max_copies,
return count;
 }
 
+static __rte_always_inline int
+__idxd_enqueue_copy(int dev_id, rte_iova_t src, rte_iova_t dst,
+   unsigned int length, uintptr_t src_hdl, uintptr_t dst_hdl,
+   int fence __rte_unused)
+{
+   struct rte_idxd_rawdev *idxd = rte_rawdevs[dev_id].dev_private;
+   struct rte_idxd_desc_batch *b = &idxd->batch_ring[idxd->next_batch];
+   uint32_t op_flags = (idxd_op_memmove << IDXD_CMD_OP_SHIFT) |
+   IDXD_FLAG_CACHE_CONTROL;
+
+   /* check for room in the handle ring */
+   if (((idxd->next_free_hdl + 1) & (idxd->hdl_ring_sz - 1)) == 
idxd->next_ret_hdl) {
+   rte_errno = ENOSPC;
+   return 0;
+   }
+   if (b->op_count >= BATCH_SIZE) {
+   rte_errno = ENOSPC;
+   return 0;
+   }
+   /* check that we can actually use the current batch */
+   if (b->submitted) {
+   rte_errno = ENOSPC;
+   return 0;
+   }
+
+   /* write the descriptor */
+   b->ops[b->op_count++] = (struct rte_idxd_hw_desc){
+   .op_flags =  op_flags,
+  

[dpdk-dev] [PATCH v2 14/18] raw/ioat: add info function for idxd devices

[Bruce Richardson]

Add the info get function for DSA devices, returning just the ring size
info about the device, same as is returned for existing IOAT/CBDMA devices.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_pci.c |  1 +
 drivers/raw/ioat/idxd_vdev.c|  1 +
 drivers/raw/ioat/ioat_common.c  | 18 ++
 drivers/raw/ioat/ioat_private.h |  3 +++
 4 files changed, 23 insertions(+)

diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c
index 13a0a03211..1ae20bc04f 100644
--- a/drivers/raw/ioat/idxd_pci.c
+++ b/drivers/raw/ioat/idxd_pci.c
@@ -105,6 +105,7 @@ static const struct rte_rawdev_ops idxd_pci_ops = {
.dev_configure = idxd_dev_configure,
.dev_start = idxd_pci_dev_start,
.dev_stop = idxd_pci_dev_stop,
+   .dev_info_get = idxd_dev_info_get,
 };
 
 /* each portal uses 4 x 4k pages */
diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c
index 73cb5d938f..3d6aa31f48 100644
--- a/drivers/raw/ioat/idxd_vdev.c
+++ b/drivers/raw/ioat/idxd_vdev.c
@@ -34,6 +34,7 @@ static const struct rte_rawdev_ops idxd_vdev_ops = {
.dev_selftest = idxd_rawdev_test,
.dump = idxd_dev_dump,
.dev_configure = idxd_dev_configure,
+   .dev_info_get = idxd_dev_info_get,
 };
 
 static void *
diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c
index 23a9d71dd3..4c225762bd 100644
--- a/drivers/raw/ioat/ioat_common.c
+++ b/drivers/raw/ioat/ioat_common.c
@@ -37,6 +37,24 @@ idxd_dev_dump(struct rte_rawdev *dev, FILE *f)
return 0;
 }
 
+int
+idxd_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info,
+   size_t info_size)
+{
+   struct rte_ioat_rawdev_config *cfg = dev_info;
+   struct idxd_rawdev *idxd = dev->dev_private;
+   struct rte_idxd_rawdev *rte_idxd = &idxd->public;
+
+   if (info_size != sizeof(*cfg))
+   return -EINVAL;
+
+   if (cfg != NULL) {
+   cfg->ring_size = rte_idxd->hdl_ring_sz;
+   cfg->hdls_disable = rte_idxd->hdls_disable;
+   }
+   return 0;
+}
+
 int
 idxd_dev_configure(const struct rte_rawdev *dev,
rte_rawdev_obj_t config, size_t config_size)
diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h
index 928c9b497c..0b17a8646b 100644
--- a/drivers/raw/ioat/ioat_private.h
+++ b/drivers/raw/ioat/ioat_private.h
@@ -60,6 +60,9 @@ extern int idxd_rawdev_create(const char *name, struct 
rte_device *dev,
 extern int idxd_dev_configure(const struct rte_rawdev *dev,
rte_rawdev_obj_t config, size_t config_size);
 
+extern int idxd_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info,
+   size_t info_size);
+
 extern int idxd_rawdev_test(uint16_t dev_id);
 
 extern int idxd_dev_dump(struct rte_rawdev *dev, FILE *f);
-- 
2.25.1

[dpdk-dev] [PATCH v2 17/18] raw/ioat: add xstats tracking for idxd devices

[Bruce Richardson]

Add update of the relevant stats for the data path functions and point the
overall device struct xstats function pointers to the existing ioat
functions.

At this point, all necessary hooks for supporting the existing unit tests
are in place so call them for each device.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_pci.c|  3 +++
 drivers/raw/ioat/idxd_vdev.c   |  3 +++
 drivers/raw/ioat/ioat_rawdev_test.c|  2 +-
 drivers/raw/ioat/rte_ioat_rawdev_fns.h | 30 +++---
 4 files changed, 25 insertions(+), 13 deletions(-)

diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c
index 1ae20bc04f..4b97b5b5fd 100644
--- a/drivers/raw/ioat/idxd_pci.c
+++ b/drivers/raw/ioat/idxd_pci.c
@@ -106,6 +106,9 @@ static const struct rte_rawdev_ops idxd_pci_ops = {
.dev_start = idxd_pci_dev_start,
.dev_stop = idxd_pci_dev_stop,
.dev_info_get = idxd_dev_info_get,
+   .xstats_get = ioat_xstats_get,
+   .xstats_get_names = ioat_xstats_get_names,
+   .xstats_reset = ioat_xstats_reset,
 };
 
 /* each portal uses 4 x 4k pages */
diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c
index 3d6aa31f48..febc5919f4 100644
--- a/drivers/raw/ioat/idxd_vdev.c
+++ b/drivers/raw/ioat/idxd_vdev.c
@@ -35,6 +35,9 @@ static const struct rte_rawdev_ops idxd_vdev_ops = {
.dump = idxd_dev_dump,
.dev_configure = idxd_dev_configure,
.dev_info_get = idxd_dev_info_get,
+   .xstats_get = ioat_xstats_get,
+   .xstats_get_names = ioat_xstats_get_names,
+   .xstats_reset = ioat_xstats_reset,
 };
 
 static void *
diff --git a/drivers/raw/ioat/ioat_rawdev_test.c 
b/drivers/raw/ioat/ioat_rawdev_test.c
index 082b3091c4..db10178871 100644
--- a/drivers/raw/ioat/ioat_rawdev_test.c
+++ b/drivers/raw/ioat/ioat_rawdev_test.c
@@ -274,5 +274,5 @@ int
 idxd_rawdev_test(uint16_t dev_id)
 {
rte_rawdev_dump(dev_id, stdout);
-   return 0;
+   return ioat_rawdev_test(dev_id);
 }
diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h 
b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
index 66e3f1a836..db8608fa6b 100644
--- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h
+++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
@@ -182,6 +182,8 @@ struct rte_idxd_user_hdl {
  */
 struct rte_idxd_rawdev {
enum rte_ioat_dev_type type;
+   struct rte_ioat_xstats xstats;
+
void *portal; /* address to write the batch descriptor */
 
/* counters to track the batches and the individual op handles */
@@ -330,19 +332,15 @@ __idxd_enqueue_copy(int dev_id, rte_iova_t src, 
rte_iova_t dst,
IDXD_FLAG_CACHE_CONTROL;
 
/* check for room in the handle ring */
-   if (((idxd->next_free_hdl + 1) & (idxd->hdl_ring_sz - 1)) == 
idxd->next_ret_hdl) {
-   rte_errno = ENOSPC;
-   return 0;
-   }
-   if (b->op_count >= BATCH_SIZE) {
-   rte_errno = ENOSPC;
-   return 0;
-   }
+   if (((idxd->next_free_hdl + 1) & (idxd->hdl_ring_sz - 1)) == 
idxd->next_ret_hdl)
+   goto failed;
+
+   if (b->op_count >= BATCH_SIZE)
+   goto failed;
+
/* check that we can actually use the current batch */
-   if (b->submitted) {
-   rte_errno = ENOSPC;
-   return 0;
-   }
+   if (b->submitted)
+   goto failed;
 
/* write the descriptor */
b->ops[b->op_count++] = (struct rte_idxd_hw_desc){
@@ -361,7 +359,13 @@ __idxd_enqueue_copy(int dev_id, rte_iova_t src, rte_iova_t 
dst,
if (++idxd->next_free_hdl == idxd->hdl_ring_sz)
idxd->next_free_hdl = 0;
 
+   idxd->xstats.enqueued++;
return 1;
+
+failed:
+   idxd->xstats.enqueue_failed++;
+   rte_errno = ENOSPC;
+   return 0;
 }
 
 static __rte_always_inline void
@@ -388,6 +392,7 @@ __idxd_perform_ops(int dev_id)
 
if (++idxd->next_batch == idxd->batch_ring_sz)
idxd->next_batch = 0;
+   idxd->xstats.started = idxd->xstats.enqueued;
 }
 
 static __rte_always_inline int
@@ -424,6 +429,7 @@ __idxd_completed_ops(int dev_id, uint8_t max_ops,
 
idxd->next_ret_hdl = h_idx;
 
+   idxd->xstats.completed += n;
return n;
 }
 
-- 
2.25.1

[dpdk-dev] [PATCH v2 16/18] raw/ioat: move xstats functions to common file

[Bruce Richardson]

The xstats functions can be used by all ioat devices so move them from the
ioat_rawdev.c file to ioat_common.c, and add the function prototypes to the
internal header file.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/ioat_common.c  | 59 +
 drivers/raw/ioat/ioat_private.h | 10 ++
 drivers/raw/ioat/ioat_rawdev.c  | 58 
 3 files changed, 69 insertions(+), 58 deletions(-)

diff --git a/drivers/raw/ioat/ioat_common.c b/drivers/raw/ioat/ioat_common.c
index 4c225762bd..08ec5a458e 100644
--- a/drivers/raw/ioat/ioat_common.c
+++ b/drivers/raw/ioat/ioat_common.c
@@ -4,9 +4,68 @@
 
 #include 
 #include 
+#include 
 
 #include "ioat_private.h"
 
+static const char * const xstat_names[] = {
+   "failed_enqueues", "successful_enqueues",
+   "copies_started", "copies_completed"
+};
+
+int
+ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[],
+   uint64_t values[], unsigned int n)
+{
+   const struct rte_ioat_rawdev *ioat = dev->dev_private;
+   const uint64_t *stats = (const void *)&ioat->xstats;
+   unsigned int i;
+
+   for (i = 0; i < n; i++) {
+   if (ids[i] > sizeof(ioat->xstats)/sizeof(*stats))
+   values[i] = 0;
+   else
+   values[i] = stats[ids[i]];
+   }
+   return n;
+}
+
+int
+ioat_xstats_get_names(const struct rte_rawdev *dev,
+   struct rte_rawdev_xstats_name *names,
+   unsigned int size)
+{
+   unsigned int i;
+
+   RTE_SET_USED(dev);
+   if (size < RTE_DIM(xstat_names))
+   return RTE_DIM(xstat_names);
+
+   for (i = 0; i < RTE_DIM(xstat_names); i++)
+   strlcpy(names[i].name, xstat_names[i], sizeof(names[i]));
+
+   return RTE_DIM(xstat_names);
+}
+
+int
+ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, uint32_t nb_ids)
+{
+   struct rte_ioat_rawdev *ioat = dev->dev_private;
+   uint64_t *stats = (void *)&ioat->xstats;
+   unsigned int i;
+
+   if (!ids) {
+   memset(&ioat->xstats, 0, sizeof(ioat->xstats));
+   return 0;
+   }
+
+   for (i = 0; i < nb_ids; i++)
+   if (ids[i] < sizeof(ioat->xstats)/sizeof(*stats))
+   stats[ids[i]] = 0;
+
+   return 0;
+}
+
 int
 idxd_dev_dump(struct rte_rawdev *dev, FILE *f)
 {
diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h
index 0b17a8646b..f4e2982e2b 100644
--- a/drivers/raw/ioat/ioat_private.h
+++ b/drivers/raw/ioat/ioat_private.h
@@ -53,6 +53,16 @@ struct idxd_rawdev {
} u;
 };
 
+int ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[],
+   uint64_t values[], unsigned int n);
+
+int ioat_xstats_get_names(const struct rte_rawdev *dev,
+   struct rte_rawdev_xstats_name *names,
+   unsigned int size);
+
+int ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids,
+   uint32_t nb_ids);
+
 extern int idxd_rawdev_create(const char *name, struct rte_device *dev,
   const struct idxd_rawdev *idxd,
   const struct rte_rawdev_ops *ops);
diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c
index 064eb839cf..385917db29 100644
--- a/drivers/raw/ioat/ioat_rawdev.c
+++ b/drivers/raw/ioat/ioat_rawdev.c
@@ -122,64 +122,6 @@ ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t 
dev_info,
return 0;
 }
 
-static const char * const xstat_names[] = {
-   "failed_enqueues", "successful_enqueues",
-   "copies_started", "copies_completed"
-};
-
-static int
-ioat_xstats_get(const struct rte_rawdev *dev, const unsigned int ids[],
-   uint64_t values[], unsigned int n)
-{
-   const struct rte_ioat_rawdev *ioat = dev->dev_private;
-   const uint64_t *stats = (const void *)&ioat->xstats;
-   unsigned int i;
-
-   for (i = 0; i < n; i++) {
-   if (ids[i] < sizeof(ioat->xstats)/sizeof(*stats))
-   values[i] = stats[ids[i]];
-   else
-   values[i] = 0;
-   }
-   return n;
-}
-
-static int
-ioat_xstats_get_names(const struct rte_rawdev *dev,
-   struct rte_rawdev_xstats_name *names,
-   unsigned int size)
-{
-   unsigned int i;
-
-   RTE_SET_USED(dev);
-   if (size < RTE_DIM(xstat_names))
-   return RTE_DIM(xstat_names);
-
-   for (i = 0; i < RTE_DIM(xstat_names); i++)
-   strlcpy(names[i].name, xstat_names[i], sizeof(names[i]));
-
-   return RTE_DIM(xstat_names);
-}
-
-static int
-ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, uint32_t nb_ids)
-{
-   struct rte_ioat_rawdev *ioat = dev->dev_private;
-   uint64_t *stats = (void *)&ioat->xstats;
-   unsigned int i;
-
-   if (!ids) {
-   memset(&ioat

[dpdk-dev] [PATCH v2 15/18] raw/ioat: create separate statistics structure

[Bruce Richardson]

Rather than having the xstats as fields inside the main driver structure,
create a separate structure type for them.

As part of the change, when updating the stats functions referring to the
stats by the old path, we can simplify them to use the id to directly index
into the stats structure, making the code shorter and simpler.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/ioat_rawdev.c | 40 +++---
 drivers/raw/ioat/rte_ioat_rawdev_fns.h | 30 ---
 2 files changed, 29 insertions(+), 41 deletions(-)

diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c
index 15133737b9..064eb839cf 100644
--- a/drivers/raw/ioat/ioat_rawdev.c
+++ b/drivers/raw/ioat/ioat_rawdev.c
@@ -132,16 +132,14 @@ ioat_xstats_get(const struct rte_rawdev *dev, const 
unsigned int ids[],
uint64_t values[], unsigned int n)
 {
const struct rte_ioat_rawdev *ioat = dev->dev_private;
+   const uint64_t *stats = (const void *)&ioat->xstats;
unsigned int i;
 
for (i = 0; i < n; i++) {
-   switch (ids[i]) {
-   case 0: values[i] = ioat->enqueue_failed; break;
-   case 1: values[i] = ioat->enqueued; break;
-   case 2: values[i] = ioat->started; break;
-   case 3: values[i] = ioat->completed; break;
-   default: values[i] = 0; break;
-   }
+   if (ids[i] < sizeof(ioat->xstats)/sizeof(*stats))
+   values[i] = stats[ids[i]];
+   else
+   values[i] = 0;
}
return n;
 }
@@ -167,35 +165,17 @@ static int
 ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids, uint32_t nb_ids)
 {
struct rte_ioat_rawdev *ioat = dev->dev_private;
+   uint64_t *stats = (void *)&ioat->xstats;
unsigned int i;
 
if (!ids) {
-   ioat->enqueue_failed = 0;
-   ioat->enqueued = 0;
-   ioat->started = 0;
-   ioat->completed = 0;
+   memset(&ioat->xstats, 0, sizeof(ioat->xstats));
return 0;
}
 
-   for (i = 0; i < nb_ids; i++) {
-   switch (ids[i]) {
-   case 0:
-   ioat->enqueue_failed = 0;
-   break;
-   case 1:
-   ioat->enqueued = 0;
-   break;
-   case 2:
-   ioat->started = 0;
-   break;
-   case 3:
-   ioat->completed = 0;
-   break;
-   default:
-   IOAT_PMD_WARN("Invalid xstat id - cannot reset value");
-   break;
-   }
-   }
+   for (i = 0; i < nb_ids; i++)
+   if (ids[i] < sizeof(ioat->xstats)/sizeof(*stats))
+   stats[ids[i]] = 0;
 
return 0;
 }
diff --git a/drivers/raw/ioat/rte_ioat_rawdev_fns.h 
b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
index 1950c8..66e3f1a836 100644
--- a/drivers/raw/ioat/rte_ioat_rawdev_fns.h
+++ b/drivers/raw/ioat/rte_ioat_rawdev_fns.h
@@ -47,17 +47,31 @@ enum rte_ioat_dev_type {
RTE_IDXD_DEV,
 };
 
+/**
+ * @internal
+ * some statistics for tracking, if added/changed update xstats fns
+ */
+struct rte_ioat_xstats {
+   uint64_t enqueue_failed;
+   uint64_t enqueued;
+   uint64_t started;
+   uint64_t completed;
+};
+
 /**
  * @internal
  * Structure representing an IOAT device instance
  */
 struct rte_ioat_rawdev {
+   /* common fields at the top - match those in rte_idxd_rawdev */
enum rte_ioat_dev_type type;
+   struct rte_ioat_xstats xstats;
+
struct rte_rawdev *rawdev;
const struct rte_memzone *mz;
const struct rte_memzone *desc_mz;
 
-   volatile uint16_t *doorbell;
+   volatile uint16_t *doorbell __rte_cache_aligned;
phys_addr_t status_addr;
phys_addr_t ring_addr;
 
@@ -70,12 +84,6 @@ struct rte_ioat_rawdev {
unsigned short next_read;
unsigned short next_write;
 
-   /* some statistics for tracking, if added/changed update xstats fns*/
-   uint64_t enqueue_failed __rte_cache_aligned;
-   uint64_t enqueued;
-   uint64_t started;
-   uint64_t completed;
-
/* to report completions, the device will write status back here */
volatile uint64_t status __rte_cache_aligned;
 
@@ -207,7 +215,7 @@ __ioat_enqueue_copy(int dev_id, phys_addr_t src, 
phys_addr_t dst,
struct rte_ioat_generic_hw_desc *desc;
 
if (space == 0) {
-   ioat->enqueue_failed++;
+   ioat->xstats.enqueue_failed++;
return 0;
}
 
@@ -226,7 +234,7 @@ __ioat_enqueue_copy(int dev_id, phys_addr_t src, 
phys_addr_t dst,
(int64_t)src_hdl);
rte_prefetch0(&ioat->desc_ring[ioat->next_write & mask]);
 
-   ioat->enque

[dpdk-dev] [PATCH v2 18/18] raw/ioat: clean up use of common test function

[Bruce Richardson]

Now that all devices can pass the same set of unit tests, elminate the
temporary idxd_rawdev_test function and move the prototype for
ioat_rawdev_test to the proper internal header file, to be used by all
device instances.

Signed-off-by: Bruce Richardson 
---
 drivers/raw/ioat/idxd_pci.c | 2 +-
 drivers/raw/ioat/idxd_vdev.c| 2 +-
 drivers/raw/ioat/ioat_private.h | 4 ++--
 drivers/raw/ioat/ioat_rawdev.c  | 2 --
 drivers/raw/ioat/ioat_rawdev_test.c | 7 ---
 5 files changed, 4 insertions(+), 13 deletions(-)

diff --git a/drivers/raw/ioat/idxd_pci.c b/drivers/raw/ioat/idxd_pci.c
index 4b97b5b5fd..926ca39ca8 100644
--- a/drivers/raw/ioat/idxd_pci.c
+++ b/drivers/raw/ioat/idxd_pci.c
@@ -100,7 +100,7 @@ idxd_pci_dev_start(struct rte_rawdev *dev)
 }
 
 static const struct rte_rawdev_ops idxd_pci_ops = {
-   .dev_selftest = idxd_rawdev_test,
+   .dev_selftest = ioat_rawdev_test,
.dump = idxd_dev_dump,
.dev_configure = idxd_dev_configure,
.dev_start = idxd_pci_dev_start,
diff --git a/drivers/raw/ioat/idxd_vdev.c b/drivers/raw/ioat/idxd_vdev.c
index febc5919f4..5529760aa2 100644
--- a/drivers/raw/ioat/idxd_vdev.c
+++ b/drivers/raw/ioat/idxd_vdev.c
@@ -31,7 +31,7 @@ struct idxd_vdev_args {
 };
 
 static const struct rte_rawdev_ops idxd_vdev_ops = {
-   .dev_selftest = idxd_rawdev_test,
+   .dev_selftest = ioat_rawdev_test,
.dump = idxd_dev_dump,
.dev_configure = idxd_dev_configure,
.dev_info_get = idxd_dev_info_get,
diff --git a/drivers/raw/ioat/ioat_private.h b/drivers/raw/ioat/ioat_private.h
index f4e2982e2b..2925483ffb 100644
--- a/drivers/raw/ioat/ioat_private.h
+++ b/drivers/raw/ioat/ioat_private.h
@@ -63,6 +63,8 @@ int ioat_xstats_get_names(const struct rte_rawdev *dev,
 int ioat_xstats_reset(struct rte_rawdev *dev, const uint32_t *ids,
uint32_t nb_ids);
 
+extern int ioat_rawdev_test(uint16_t dev_id);
+
 extern int idxd_rawdev_create(const char *name, struct rte_device *dev,
   const struct idxd_rawdev *idxd,
   const struct rte_rawdev_ops *ops);
@@ -73,8 +75,6 @@ extern int idxd_dev_configure(const struct rte_rawdev *dev,
 extern int idxd_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t dev_info,
size_t info_size);
 
-extern int idxd_rawdev_test(uint16_t dev_id);
-
 extern int idxd_dev_dump(struct rte_rawdev *dev, FILE *f);
 
 #endif /* _IOAT_PRIVATE_H_ */
diff --git a/drivers/raw/ioat/ioat_rawdev.c b/drivers/raw/ioat/ioat_rawdev.c
index 385917db29..06af235f96 100644
--- a/drivers/raw/ioat/ioat_rawdev.c
+++ b/drivers/raw/ioat/ioat_rawdev.c
@@ -122,8 +122,6 @@ ioat_dev_info_get(struct rte_rawdev *dev, rte_rawdev_obj_t 
dev_info,
return 0;
 }
 
-extern int ioat_rawdev_test(uint16_t dev_id);
-
 static int
 ioat_rawdev_create(const char *name, struct rte_pci_device *dev)
 {
diff --git a/drivers/raw/ioat/ioat_rawdev_test.c 
b/drivers/raw/ioat/ioat_rawdev_test.c
index db10178871..534b07b124 100644
--- a/drivers/raw/ioat/ioat_rawdev_test.c
+++ b/drivers/raw/ioat/ioat_rawdev_test.c
@@ -269,10 +269,3 @@ ioat_rawdev_test(uint16_t dev_id)
free(ids);
return -1;
 }
-
-int
-idxd_rawdev_test(uint16_t dev_id)
-{
-   rte_rawdev_dump(dev_id, stdout);
-   return ioat_rawdev_test(dev_id);
-}
-- 
2.25.1

Re: [dpdk-dev] [PATCH v2 00/18] raw/ioat: enhancements and new hardware support

[Bruce Richardson]

On Fri, Aug 21, 2020 at 05:29:26PM +0100, Bruce Richardson wrote:
> This patchset adds some small enhancements, some rework and also support
> for new hardware to the ioat rawdev driver. Most rework and enhancements
> are largely self-explanatory from the individual patches.
> 
> The new hardware support is for the Intel(R) DSA accelerator which will be
> present in future Intel processors. A description of this new hardware is
> covered in [1]. Functions specific to the new hardware use the "idxd"
> prefix, for consistency with the kernel driver. *** SUBJECT HERE ***
> 
> [1] https://01.org/blogs/2019/introducing-intel-data-streaming-accelerator
> 
> ---
> V2:
>  * Included documentation additions in the set
>  * Split off the rawdev unit test changes to a separate patchset for easier
>review
>  * General code improvements and cleanups
> 
> 
Forgot to add that this patchset is based on top of the proposed rawdev API
changes:
http://patches.dpdk.org/project/dpdk/list/?series=11639

[dpdk-dev] [PATCH v2 0/4] fixes for example app builds

[Bruce Richardson]

While helping prepare some patches for converting the DPDK examples to
build using pkg-config, a number of errors and warnings were encountered in
the example app builds. These patches fix those issues.

NOTE: no makefile changes are included since make support is being removed
in this release

Bruce Richardson (4):
  power: make guest channel headers public
  examples/vm_power_manager: fix string truncation warning
  examples/mp_server: fix snprintf overflow
  examples/mp_server: clear string truncation warning

 .../client_server_mp/mp_server/main.c   | 17 ++---
 .../client_server_mp/shared/common.h|  2 +-
 examples/vm_power_manager/channel_manager.c | 11 ---
 examples/vm_power_manager/channel_monitor.c |  1 -
 examples/vm_power_manager/channel_monitor.h |  2 +-
 examples/vm_power_manager/guest_cli/main.c  |  1 +
 .../guest_cli/vm_power_cli_guest.c  |  2 +-
 .../guest_cli/vm_power_cli_guest.h  |  2 +-
 examples/vm_power_manager/vm_power_cli.c|  1 -
 lib/librte_power/guest_channel.c|  3 +--
 lib/librte_power/meson.build|  4 +++-
 lib/librte_power/power_kvm_vm.c |  3 +--
 ..._commands.h => rte_power_channel_commands.h} |  0
 ...uest_channel.h => rte_power_guest_channel.h} |  2 +-
 14 files changed, 29 insertions(+), 22 deletions(-)
 rename lib/librte_power/{channel_commands.h => rte_power_channel_commands.h} 
(100%)
 rename lib/librte_power/{guest_channel.h => rte_power_guest_channel.h} (98%)

-- 
2.25.1

[dpdk-dev] [PATCH v2 1/4] power: make guest channel headers public

[Bruce Richardson]

The channel commands header file contains definitions that are used by the
example application for power management, and so need to be made public.
Similarly the guest_channel.h header is used by the guest_cli example
utility, so needs to be public also.  Without this change, the example
apps, or any end-user apps based on them, can not be compiled outside the
main DPDK build.

Fixes: 210c383e247b ("power: packet format for vm power management")
Fixes: cd0d5547e873 ("power: vm communication channels in guest")
Cc: sta...@dpdk.org

Signed-off-by: Bruce Richardson 
---
 examples/vm_power_manager/channel_manager.c   | 1 -
 examples/vm_power_manager/channel_monitor.c   | 1 -
 examples/vm_power_manager/channel_monitor.h   | 2 +-
 examples/vm_power_manager/guest_cli/main.c| 1 +
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.c  | 2 +-
 examples/vm_power_manager/guest_cli/vm_power_cli_guest.h  | 2 +-
 examples/vm_power_manager/vm_power_cli.c  | 1 -
 lib/librte_power/guest_channel.c  | 3 +--
 lib/librte_power/meson.build  | 4 +++-
 lib/librte_power/power_kvm_vm.c   | 3 +--
 .../{channel_commands.h => rte_power_channel_commands.h}  | 0
 .../{guest_channel.h => rte_power_guest_channel.h}| 2 +-
 12 files changed, 10 insertions(+), 12 deletions(-)
 rename lib/librte_power/{channel_commands.h => rte_power_channel_commands.h} 
(100%)
 rename lib/librte_power/{guest_channel.h => rte_power_guest_channel.h} (98%)

diff --git a/examples/vm_power_manager/channel_manager.c 
b/examples/vm_power_manager/channel_manager.c
index 74a2a677e8..3da01b46d8 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -27,7 +27,6 @@
 #include 
 
 #include "channel_manager.h"
-#include "channel_commands.h"
 #include "channel_monitor.h"
 #include "power_manager.h"
 
diff --git a/examples/vm_power_manager/channel_monitor.c 
b/examples/vm_power_manager/channel_monitor.c
index 1d00a6cf6c..75a29d2589 100644
--- a/examples/vm_power_manager/channel_monitor.c
+++ b/examples/vm_power_manager/channel_monitor.c
@@ -35,7 +35,6 @@
 
 #include 
 #include "channel_monitor.h"
-#include "channel_commands.h"
 #include "channel_manager.h"
 #include "power_manager.h"
 #include "oob_monitor.h"
diff --git a/examples/vm_power_manager/channel_monitor.h 
b/examples/vm_power_manager/channel_monitor.h
index 7362a80d26..fe6ceedc95 100644
--- a/examples/vm_power_manager/channel_monitor.h
+++ b/examples/vm_power_manager/channel_monitor.h
@@ -6,7 +6,7 @@
 #define CHANNEL_MONITOR_H_
 
 #include "channel_manager.h"
-#include "channel_commands.h"
+#include "rte_power_channel_commands.h"
 
 struct core_share {
unsigned int pcpu;
diff --git a/examples/vm_power_manager/guest_cli/main.c 
b/examples/vm_power_manager/guest_cli/main.c
index f63b3c988a..4c0c98f498 100644
--- a/examples/vm_power_manager/guest_cli/main.c
+++ b/examples/vm_power_manager/guest_cli/main.c
@@ -14,6 +14,7 @@
 #include 
 #include 
 
+#include "rte_power_channel_commands.h"
 #include "vm_power_cli_guest.h"
 #include "parse.h"
 
diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c 
b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
index 96c1a1ff69..263e7ec9ff 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.c
@@ -19,7 +19,7 @@
 #include 
 
 #include 
-#include 
+#include 
 
 #include "vm_power_cli_guest.h"
 
diff --git a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.h 
b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.h
index 6ad14a3dea..33c01ff7ca 100644
--- a/examples/vm_power_manager/guest_cli/vm_power_cli_guest.h
+++ b/examples/vm_power_manager/guest_cli/vm_power_cli_guest.h
@@ -9,7 +9,7 @@
 extern "C" {
 #endif
 
-#include "channel_commands.h"
+struct channel_packet;
 
 struct channel_packet *get_policy(void);
 
diff --git a/examples/vm_power_manager/vm_power_cli.c 
b/examples/vm_power_manager/vm_power_cli.c
index 7edeaccda5..4c41645664 100644
--- a/examples/vm_power_manager/vm_power_cli.c
+++ b/examples/vm_power_manager/vm_power_cli.c
@@ -21,7 +21,6 @@
 #include "channel_manager.h"
 #include "channel_monitor.h"
 #include "power_manager.h"
-#include "channel_commands.h"
 
 struct cmd_quit_result {
cmdline_fixed_string_t quit;
diff --git a/lib/librte_power/guest_channel.c b/lib/librte_power/guest_channel.c
index 7b5926e5c4..d49cf0cd57 100644
--- a/lib/librte_power/guest_channel.c
+++ b/lib/librte_power/guest_channel.c
@@ -16,8 +16,7 @@
 
 #include 
 
-#include "guest_channel.h"
-#include "channel_commands.h"
+#include "rte_power_guest_channel.h"
 
 #define RTE_LOGTYPE_GUEST_CHANNEL RTE_LOGTYPE_USER1
 
diff --git a/lib/librte_power/meson.build b/lib/librte_power/meson.build
index 78c031c943..e9b1fa6e45 100644
---

[dpdk-dev] [PATCH v2 3/4] examples/mp_server: fix snprintf overflow

[Bruce Richardson]

When producing a printable mac address the buffer was appropriately sized
for holding the mac address exactly, but the actual snprintf included a
'\n' character at the end, which means that the snprintf technically is
getting truncated i.e. the \n would not be added due to lack of space.
This gets flagged as a problem by modern versions of gcc, e.g. on Ubuntu
20.04.

main.c:77:37: warning: ‘__builtin___snprintf_chk’ output truncated before the 
last format character [-Wformat-truncation=]
   77 | "%02x:%02x:%02x:%02x:%02x:%02x\n",
  | ^

Since the \n is getting stripped anyway, we can fix the issue by just
removing it. In the process we can switch to using the standard ethernet
address formating function from rte_ether.h.

Fixes: af75078fece3 ("first public release")
Cc: sta...@dpdk.org
Cc: Stephen Hemminger 

Signed-off-by: Bruce Richardson 

---
V2: switched code to use standard formatting function
---
 .../client_server_mp/mp_server/main.c   | 17 ++---
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/examples/multi_process/client_server_mp/mp_server/main.c 
b/examples/multi_process/client_server_mp/mp_server/main.c
index 280dab8672..fb2aa49678 100644
--- a/examples/multi_process/client_server_mp/mp_server/main.c
+++ b/examples/multi_process/client_server_mp/mp_server/main.c
@@ -59,12 +59,17 @@ static struct client_rx_buf *cl_rx_buf;
 static const char *
 get_printable_mac_addr(uint16_t port)
 {
-   static const char err_address[] = "00:00:00:00:00:00";
-   static char addresses[RTE_MAX_ETHPORTS][sizeof(err_address)];
+   static const struct rte_ether_addr null_mac; /* static defaults to 0 */
+   static char err_address[32];
+   static char addresses[RTE_MAX_ETHPORTS][32];
int ret;
 
-   if (unlikely(port >= RTE_MAX_ETHPORTS))
+   if (unlikely(port >= RTE_MAX_ETHPORTS)) {
+   if (err_address[0] == '\0')
+   rte_ether_format_addr(err_address,
+   sizeof(err_address), &null_mac);
return err_address;
+   }
if (unlikely(addresses[port][0]=='\0')){
struct rte_ether_addr mac;
ret = rte_eth_macaddr_get(port, &mac);
@@ -73,10 +78,8 @@ get_printable_mac_addr(uint16_t port)
   port, rte_strerror(-ret));
return err_address;
}
-   snprintf(addresses[port], sizeof(addresses[port]),
-   "%02x:%02x:%02x:%02x:%02x:%02x\n",
-   mac.addr_bytes[0], mac.addr_bytes[1], 
mac.addr_bytes[2],
-   mac.addr_bytes[3], mac.addr_bytes[4], 
mac.addr_bytes[5]);
+   rte_ether_format_addr(addresses[port],
+   sizeof(addresses[port]), &mac);
}
return addresses[port];
 }
-- 
2.25.1

[dpdk-dev] [PATCH v2 2/4] examples/vm_power_manager: fix string truncation warning

[Bruce Richardson]

When compiling on ubuntu 20.04, a warning was issued about possible
truncation of the path string for the power management socket.

channel_manager.c: In function ‘add_all_channels’:
channel_manager.c:470:41: warning: ‘%s’ directive output may be truncated 
writing up to 255 bytes into a region of size 90 [-Wformat-truncation=]
  470 | sizeof(chan_info->channel_path), "%s%s",
  | ^~

This can be fixed by adding in an explicit truncation check to the code and
handling it appropriately.

Fixes: e8ae9b662506 ("examples/vm_power: channel manager and monitor in host")
Cc: sta...@dpdk.org

Signed-off-by: Bruce Richardson 
---
 examples/vm_power_manager/channel_manager.c | 10 --
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/examples/vm_power_manager/channel_manager.c 
b/examples/vm_power_manager/channel_manager.c
index 3da01b46d8..0a28cb643b 100644
--- a/examples/vm_power_manager/channel_manager.c
+++ b/examples/vm_power_manager/channel_manager.c
@@ -466,9 +466,15 @@ add_all_channels(const char *vm_name)
continue;
}
 
-   snprintf(chan_info->channel_path,
+   if ((size_t)snprintf(chan_info->channel_path,
sizeof(chan_info->channel_path), "%s%s",
-   CHANNEL_MGR_SOCKET_PATH, dir->d_name);
+   CHANNEL_MGR_SOCKET_PATH, dir->d_name)
+   >= sizeof(chan_info->channel_path)) {
+   RTE_LOG(ERR, CHANNEL_MANAGER, "Pathname too long for 
channel '%s%s'\n",
+   CHANNEL_MGR_SOCKET_PATH, dir->d_name);
+   rte_free(chan_info);
+   continue;
+   }
 
if (setup_channel_info(&vm_info, &chan_info, channel_num) < 0) {
rte_free(chan_info);
-- 
2.25.1

[dpdk-dev] [PATCH v2 4/4] examples/mp_server: clear string truncation warning

[Bruce Richardson]

Compiling with GCC 9.3 on Ubuntu 20.04 gives a warning about possible
string truncation when getting the RX queue name:

In file included from init.c:36:
init.c: In function ‘init’:
../shared/common.h:38:28: warning: ‘%u’ directive output may be truncated 
writing between 1 and 10 bytes into a region of size 8 [-Wformat-truncation=]
   38 | #define MP_CLIENT_RXQ_NAME "MProc_Client_%u_RX"
  |^~~~
../shared/common.h:52:35: note: in expansion of macro ‘MP_CLIENT_RXQ_NAME’
   52 |  snprintf(buffer, sizeof(buffer), MP_CLIENT_RXQ_NAME, id);
  |   ^~

This is a false positive, as the value of the "id" is limited to 255, being
stored in the app as a uint8_t value, removing the possibility of the %u
being replaced by anything other then 3 characters max (rather than up to
10 as thought by the compiler). Therefore, the warning can be easily
removed by changing the type of the "id" parameter to the local function
from "unsigned" to "uint8_t" also, ensuring the compiler is aware of the
range limit.

Cc: sta...@dpdk.org

Signed-off-by: Bruce Richardson 
---
 examples/multi_process/client_server_mp/shared/common.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/examples/multi_process/client_server_mp/shared/common.h 
b/examples/multi_process/client_server_mp/shared/common.h
index 6dd43fcac2..76beca0101 100644
--- a/examples/multi_process/client_server_mp/shared/common.h
+++ b/examples/multi_process/client_server_mp/shared/common.h
@@ -43,7 +43,7 @@ struct port_info {
  * Given the rx queue name template above, get the queue name
  */
 static inline const char *
-get_rx_queue_name(unsigned id)
+get_rx_queue_name(uint8_t id)
 {
/* buffer for return value. Size calculated by %u being replaced
 * by maximum 3 digits (plus an extra byte for safety) */
-- 
2.25.1

[dpdk-dev] [PATCH] pci/linux: copy new id for inserted device

[Jim Harris]

When a device is inserted into an existing BDF slot
that has not been probed, we must overwrite the old
PCI ID with the ID of the new function. Otherwise
we may not probe the function with the correct driver,
if at all.

Signed-off-by: Jim Harris 
---
 drivers/bus/pci/linux/pci.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/drivers/bus/pci/linux/pci.c b/drivers/bus/pci/linux/pci.c
index a2198abf4..4701661e5 100644
--- a/drivers/bus/pci/linux/pci.c
+++ b/drivers/bus/pci/linux/pci.c
@@ -352,6 +352,7 @@ pci_scan_one(const char *dirname, const struct rte_pci_addr 
*addr)
if (!rte_dev_is_probed(&dev2->device)) {
dev2->kdrv = dev->kdrv;
dev2->max_vfs = dev->max_vfs;
+   memcpy(&dev2->id, &dev->id, 
sizeof(dev2->id));
pci_name_set(dev2);
memmove(dev2->mem_resource,
dev->mem_resource,
@@ -365,7 +366,8 @@ pci_scan_one(const char *dirname, const struct rte_pci_addr 
*addr)
 * need to do anything here unless...
 **/
if (dev2->kdrv != dev->kdrv ||
-   dev2->max_vfs != dev->max_vfs)
+   dev2->max_vfs != dev->max_vfs ||
+   memcmp(&dev2->id, &dev->id, 
sizeof(dev2->id))
/*
 * This should not happens.
 * But it is still possible if
-- 
2.20.1

Re: [dpdk-dev] [PATCH] cmdline: avoid name clash with Windows system types

[Ranjit Menon]

On 8/21/2020 11:48 AM, Dmitry Kozlyuk wrote:

cmdline_numtype member names clash with Windows system identifiers.
Add CMDLINE_ prefix to cmdline constants to avoid this and possible
future conflicts.

Signed-off-by: Dmitry Kozlyuk 
Suggested-by: Ranjit Menon 
---

This patch made some lines overlong. When breaking them, indentation
style is kept consistent with nearby code; it's weird in some places.

  app/test-cmdline/commands.c   |   2 +-
  app/test-pmd/bpf_cmd.c|  10 +-
  app/test-pmd/cmdline.c| 715 ++
  app/test-pmd/cmdline_flow.c   |   2 +-
  app/test-pmd/cmdline_mtr.c|  92 ++-
  app/test-pmd/cmdline_tm.c | 177 ++---
  app/test/test_cmdline_num.c   |  48 +-
  examples/ethtool/ethtool-app/ethapp.c |  19 +-
  examples/ipsec-secgw/parser.c |   2 +-
  examples/qos_sched/cmdline.c  |  46 +-
  .../guest_cli/vm_power_cli_guest.c|   2 +-
  examples/vm_power_manager/vm_power_cli.c  |   8 +-
  lib/librte_cmdline/cmdline_parse_num.c|  65 +-
  lib/librte_cmdline/cmdline_parse_num.h|  16 +-
  14 files changed, 627 insertions(+), 577 deletions(-)




Thanks for doing this, Dmitry!

Acked-by: Ranjit Menon