Reviewed-by: Chengwen Feng <fengcheng...@huawei.com>
On 2021/9/17 23:42, Conor Walsh wrote: > Add the basic device probe/remove skeleton code and initial documentation > for new IOAT DMA driver. Maintainers update is also included in this > patch. > > Signed-off-by: Conor Walsh <conor.wa...@intel.com> > Reviewed-by: Kevin Laatz <kevin.la...@intel.com> > --- > MAINTAINERS | 6 +++ > doc/guides/dmadevs/index.rst | 2 + > doc/guides/dmadevs/ioat.rst | 64 ++++++++++++++++++++++++ > doc/guides/rel_notes/release_21_11.rst | 7 +-- > drivers/dma/ioat/ioat_dmadev.c | 69 ++++++++++++++++++++++++++ > drivers/dma/ioat/ioat_hw_defs.h | 35 +++++++++++++ > drivers/dma/ioat/ioat_internal.h | 20 ++++++++ > drivers/dma/ioat/meson.build | 7 +++ > drivers/dma/ioat/version.map | 3 ++ > drivers/dma/meson.build | 1 + > 10 files changed, 211 insertions(+), 3 deletions(-) > create mode 100644 doc/guides/dmadevs/ioat.rst > create mode 100644 drivers/dma/ioat/ioat_dmadev.c > create mode 100644 drivers/dma/ioat/ioat_hw_defs.h > create mode 100644 drivers/dma/ioat/ioat_internal.h > create mode 100644 drivers/dma/ioat/meson.build > create mode 100644 drivers/dma/ioat/version.map > > diff --git a/MAINTAINERS b/MAINTAINERS > index 9cb59b831d..70993d23e8 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -1209,6 +1209,12 @@ M: Kevin Laatz <kevin.la...@intel.com> > F: drivers/dma/idxd/ > F: doc/guides/dmadevs/idxd.rst > > +Intel IOAT - EXPERIMENTAL > +M: Bruce Richardson <bruce.richard...@intel.com> > +M: Conor Walsh <conor.wa...@intel.com> > +F: drivers/dma/ioat/ > +F: doc/guides/dmadevs/ioat.rst > + > > RegEx Drivers > ------------- > diff --git a/doc/guides/dmadevs/index.rst b/doc/guides/dmadevs/index.rst > index 5d4abf880e..c59f4b5c92 100644 > --- a/doc/guides/dmadevs/index.rst > +++ b/doc/guides/dmadevs/index.rst > @@ -12,3 +12,5 @@ an application through DMA API. > :numbered: > > idxd > + ioat > + > diff --git a/doc/guides/dmadevs/ioat.rst b/doc/guides/dmadevs/ioat.rst > new file mode 100644 > index 0000000000..45a2e65d70 > --- /dev/null > +++ b/doc/guides/dmadevs/ioat.rst > @@ -0,0 +1,64 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > + Copyright(c) 2021 Intel Corporation. > + > +.. include:: <isonum.txt> > + > +IOAT DMA Device Driver > +======================= > + > +The ``ioat`` dmadev driver provides a poll-mode driver (PMD) for Intel\ > +|reg| QuickData Technology which is part of part of Intel\ |reg| I/O > +Acceleration Technology (`Intel I/OAT > +<https://www.intel.com/content/www/us/en/wireless-network/accel-technology.html>`_). > +This PMD, when used on supported hardware, allows data copies, for example, > +cloning packet data, to be accelerated by IOAT hardware rather than having to > +be done by software, freeing up CPU cycles for other tasks. > + > +Hardware Requirements > +---------------------- > + > +The ``dpdk-devbind.py`` script, included with DPDK, can be used to show the > +presence of supported hardware. Running ``dpdk-devbind.py --status-dev dma`` > +will show all the DMA devices on the system, IOAT devices are included in > this > +list. For Intel\ |reg| IOAT devices, the hardware will often be listed as > +"Crystal Beach DMA", or "CBDMA" or on some newer systems '0b00' due to the > +absence of pci-id database entries for them at this point. > + > +Compilation > +------------ > + > +For builds using ``meson`` and ``ninja``, the driver will be built when the > +target platform is x86-based. No additional compilation steps are necessary. > + > +Device Setup > +------------- > + > +Intel\ |reg| IOAT devices will need to be bound to a suitable DPDK-supported > +user-space IO driver such as ``vfio-pci`` in order to be used by DPDK. > + > +The ``dpdk-devbind.py`` script can be used to view the state of the devices > using:: > + > + $ dpdk-devbind.py --status-dev dma > + > +The ``dpdk-devbind.py`` script can also be used to bind devices to a > suitable driver. > +For example:: > + > + $ dpdk-devbind.py -b vfio-pci 00:01.0 00:01.1 > + > +Device Probing and Initialization > +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > + > +For devices bound to a suitable DPDK-supported driver (``vfio-pci``), the HW > +devices will be found as part of the device scan done at application > +initialization time without the need to pass parameters to the application. > + > +If the application does not require all the devices available an allowlist > can > +be used in the same way that other DPDK devices use them. > + > +For example:: > + > + $ dpdk-test -a <b:d:f> > + > +Once probed successfully, the device will appear as a ``dmadev``, that is a > +"DMA device type" inside DPDK, and can be accessed using APIs from the > +``rte_dmadev`` library. > diff --git a/doc/guides/rel_notes/release_21_11.rst > b/doc/guides/rel_notes/release_21_11.rst > index c0bfd9c1ba..4d2b7bde1b 100644 > --- a/doc/guides/rel_notes/release_21_11.rst > +++ b/doc/guides/rel_notes/release_21_11.rst > @@ -92,10 +92,11 @@ New Features > * Device allocation and it's multi-process support. > * Control and data plane functions. > > -* **Added IDXD dmadev driver implementation.** > +* **Added Intel dmadev driver implementations.** > > - The IDXD dmadev driver provide device drivers for the Intel DSA devices. > - This device driver can be used through the generic dmadev API. > + The IDXD and IOAT dmadev drivers provide device drivers for Intel DSA > + and IOAT devices. These device drivers can be used through the generic > + dmadev API. > > > Removed Items > diff --git a/drivers/dma/ioat/ioat_dmadev.c b/drivers/dma/ioat/ioat_dmadev.c > new file mode 100644 > index 0000000000..f3491d45b1 > --- /dev/null > +++ b/drivers/dma/ioat/ioat_dmadev.c > @@ -0,0 +1,69 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2021 Intel Corporation > + */ > + > +#include <rte_bus_pci.h> > +#include <rte_dmadev_pmd.h> > + > +#include "ioat_internal.h" > + > +static struct rte_pci_driver ioat_pmd_drv; > + > +RTE_LOG_REGISTER_DEFAULT(ioat_pmd_logtype, INFO); > + > +#define IOAT_PMD_NAME dmadev_ioat > +#define IOAT_PMD_NAME_STR RTE_STR(IOAT_PMD_NAME) > + > +/* Probe DMA device. */ > +static int > +ioat_dmadev_probe(struct rte_pci_driver *drv, struct rte_pci_device *dev) > +{ > + char name[32]; > + > + rte_pci_device_name(&dev->addr, name, sizeof(name)); > + IOAT_PMD_INFO("Init %s on NUMA node %d", name, dev->device.numa_node); > + > + dev->device.driver = &drv->driver; > + return 0; > +} > + > +/* Remove DMA device. */ > +static int > +ioat_dmadev_remove(struct rte_pci_device *dev) > +{ > + char name[32]; > + > + rte_pci_device_name(&dev->addr, name, sizeof(name)); > + > + IOAT_PMD_INFO("Closing %s on NUMA node %d", > + name, dev->device.numa_node); > + > + return 0; > +} > + > +static const struct rte_pci_id pci_id_ioat_map[] = { > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_SKX) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX0) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX1) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX2) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX3) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX4) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX5) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX6) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDX7) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXE) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_BDXF) }, > + { RTE_PCI_DEVICE(IOAT_VENDOR_ID, IOAT_DEVICE_ID_ICX) }, > + { .vendor_id = 0, /* sentinel */ }, > +}; > + > +static struct rte_pci_driver ioat_pmd_drv = { > + .id_table = pci_id_ioat_map, > + .drv_flags = RTE_PCI_DRV_NEED_MAPPING | RTE_PCI_DRV_INTR_LSC, > + .probe = ioat_dmadev_probe, > + .remove = ioat_dmadev_remove, > +}; > + > +RTE_PMD_REGISTER_PCI(IOAT_PMD_NAME, ioat_pmd_drv); > +RTE_PMD_REGISTER_PCI_TABLE(IOAT_PMD_NAME, pci_id_ioat_map); > +RTE_PMD_REGISTER_KMOD_DEP(IOAT_PMD_NAME, "* igb_uio | uio_pci_generic | > vfio-pci"); > diff --git a/drivers/dma/ioat/ioat_hw_defs.h b/drivers/dma/ioat/ioat_hw_defs.h > new file mode 100644 > index 0000000000..eeabba41ef > --- /dev/null > +++ b/drivers/dma/ioat/ioat_hw_defs.h > @@ -0,0 +1,35 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright(c) 2021 Intel Corporation > + */ > + > +#ifndef IOAT_HW_DEFS_H > +#define IOAT_HW_DEFS_H > + > +#ifdef __cplusplus > +extern "C" { > +#endif > + > +#include <stdint.h> > + > +#define IOAT_VER_3_0 0x30 > +#define IOAT_VER_3_3 0x33 > + > +#define IOAT_VENDOR_ID 0x8086 > +#define IOAT_DEVICE_ID_SKX 0x2021 > +#define IOAT_DEVICE_ID_BDX0 0x6f20 > +#define IOAT_DEVICE_ID_BDX1 0x6f21 > +#define IOAT_DEVICE_ID_BDX2 0x6f22 > +#define IOAT_DEVICE_ID_BDX3 0x6f23 > +#define IOAT_DEVICE_ID_BDX4 0x6f24 > +#define IOAT_DEVICE_ID_BDX5 0x6f25 > +#define IOAT_DEVICE_ID_BDX6 0x6f26 > +#define IOAT_DEVICE_ID_BDX7 0x6f27 > +#define IOAT_DEVICE_ID_BDXE 0x6f2E > +#define IOAT_DEVICE_ID_BDXF 0x6f2F > +#define IOAT_DEVICE_ID_ICX 0x0b00 > + > +#ifdef __cplusplus > +} > +#endif > + > +#endif /* IOAT_HW_DEFS_H */ > diff --git a/drivers/dma/ioat/ioat_internal.h > b/drivers/dma/ioat/ioat_internal.h > new file mode 100644 > index 0000000000..f1ec12a919 > --- /dev/null > +++ b/drivers/dma/ioat/ioat_internal.h > @@ -0,0 +1,20 @@ > +/* SPDX-License-Identifier: BSD-3-Clause > + * Copyright 2021 Intel Corporation > + */ > + > +#ifndef _IOAT_INTERNAL_H_ > +#define _IOAT_INTERNAL_H_ > + > +#include "ioat_hw_defs.h" > + > +extern int ioat_pmd_logtype; > + > +#define IOAT_PMD_LOG(level, fmt, args...) rte_log(RTE_LOG_ ## level, \ > + ioat_pmd_logtype, "IOAT: %s(): " fmt "\n", __func__, ##args) > + > +#define IOAT_PMD_DEBUG(fmt, args...) IOAT_PMD_LOG(DEBUG, fmt, ## args) > +#define IOAT_PMD_INFO(fmt, args...) IOAT_PMD_LOG(INFO, fmt, ## args) > +#define IOAT_PMD_ERR(fmt, args...) IOAT_PMD_LOG(ERR, fmt, ## args) > +#define IOAT_PMD_WARN(fmt, args...) IOAT_PMD_LOG(WARNING, fmt, ## args) > + > +#endif /* _IOAT_INTERNAL_H_ */ > diff --git a/drivers/dma/ioat/meson.build b/drivers/dma/ioat/meson.build > new file mode 100644 > index 0000000000..d67fac96fb > --- /dev/null > +++ b/drivers/dma/ioat/meson.build > @@ -0,0 +1,7 @@ > +# SPDX-License-Identifier: BSD-3-Clause > +# Copyright 2021 Intel Corporation > + > +build = dpdk_conf.has('RTE_ARCH_X86') > +reason = 'only supported on x86' > +sources = files('ioat_dmadev.c') > +deps += ['bus_pci', 'dmadev'] > diff --git a/drivers/dma/ioat/version.map b/drivers/dma/ioat/version.map > new file mode 100644 > index 0000000000..c2e0723b4c > --- /dev/null > +++ b/drivers/dma/ioat/version.map > @@ -0,0 +1,3 @@ > +DPDK_22 { > + local: *; > +}; > diff --git a/drivers/dma/meson.build b/drivers/dma/meson.build > index 411be7a240..a69418ce9b 100644 > --- a/drivers/dma/meson.build > +++ b/drivers/dma/meson.build > @@ -3,6 +3,7 @@ > > drivers = [ > 'idxd', > + 'ioat', > 'skeleton', > ] > std_deps = ['dmadev'] >
