On 5/9/2023 12:09 PM, Gupta, Nipun wrote: > [AMD Official Use Only - General] > > > >> -----Original Message----- >> From: Xia, Chenbo <chenbo....@intel.com> >> Sent: Tuesday, May 9, 2023 12:25 PM >> To: Gupta, Nipun <nipun.gu...@amd.com>; dev@dpdk.org; >> tho...@monjalon.net; david.march...@redhat.com >> Cc: Yigit, Ferruh <ferruh.yi...@amd.com>; Anand, Harpreet >> <harpreet.an...@amd.com>; Agarwal, Nikhil <nikhil.agar...@amd.com> >> Subject: RE: [PATCH v4 1/4] bus/cdx: introduce cdx bus >> >> Caution: This message originated from an External Source. Use proper >> caution when opening attachments, clicking links, or responding. >> >> >>> -----Original Message----- >>> From: Nipun Gupta <nipun.gu...@amd.com> >>> Sent: Monday, May 8, 2023 7:18 PM >>> To: dev@dpdk.org; tho...@monjalon.net; david.march...@redhat.com >>> Cc: ferruh.yi...@amd.com; harpreet.an...@amd.com; >> nikhil.agar...@amd.com; >>> Nipun Gupta <nipun.gu...@amd.com> >>> Subject: [PATCH v4 1/4] bus/cdx: introduce cdx bus >>> >>> CDX bus supports multiple type of devices, which can be >>> exposed to user-space via vfio-cdx. >>> >>> vfio-cdx provides the MMIO IO_MEMORY regions as well as the >>> DMA interface for the device (IOMMU). >>> >>> This support aims to enable the DPDK to support the cdx >>> devices in user-space using VFIO interface. >>> >>> Signed-off-by: Nipun Gupta <nipun.gu...@amd.com> >>> --- >>> MAINTAINERS | 5 + >>> doc/guides/rel_notes/release_23_07.rst | 6 + >>> drivers/bus/cdx/bus_cdx_driver.h | 201 ++++++++++ >>> drivers/bus/cdx/cdx.c | 520 +++++++++++++++++++++++++ >>> drivers/bus/cdx/cdx_logs.h | 37 ++ >>> drivers/bus/cdx/cdx_vfio.c | 437 +++++++++++++++++++++ >>> drivers/bus/cdx/meson.build | 13 + >>> drivers/bus/cdx/private.h | 49 +++ >>> drivers/bus/cdx/version.map | 11 + >>> drivers/bus/meson.build | 1 + >>> 10 files changed, 1280 insertions(+) >>> create mode 100644 drivers/bus/cdx/bus_cdx_driver.h >>> create mode 100644 drivers/bus/cdx/cdx.c >>> create mode 100644 drivers/bus/cdx/cdx_logs.h >>> create mode 100644 drivers/bus/cdx/cdx_vfio.c >>> create mode 100644 drivers/bus/cdx/meson.build >>> create mode 100644 drivers/bus/cdx/private.h >>> create mode 100644 drivers/bus/cdx/version.map >>> >> >> ... >> >>> --- /dev/null >>> +++ b/drivers/bus/cdx/cdx.c >>> @@ -0,0 +1,520 @@ >>> +/* SPDX-License-Identifier: BSD-3-Clause >>> + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. >>> + */ >>> + >>> +/* >>> + * Architecture Overview >>> + * ===================== >>> + * CDX is a Hardware Architecture designed for AMD FPGA devices. It >>> + * consists of sophisticated mechanism for interaction between FPGA, >>> + * Firmware and the APUs (Application CPUs). >>> + * >>> + * Firmware resides on RPU (Realtime CPUs) which interacts with >>> + * the FPGA program manager and the APUs. The RPU provides memory- >> mapped >>> + * interface (RPU if) which is used to communicate with APUs. >>> + * >>> + * The diagram below shows an overview of the CDX architecture: >>> + * >>> + * +--------------------------------------+ >>> + * | DPDK | >>> + * | DPDK CDX drivers | >>> + * | | | >>> + * | DPDK CDX bus | >>> + * | | | >>> + * +-----------------------------|--------+ >>> + * | >>> + * +-----------------------------|--------+ >>> + * | Application CPUs (APU) | | >>> + * | | | >>> + * | VFIO CDX driver | >>> + * | Linux OS | | >>> + * | Linux CDX bus | >>> + * | | | >>> + * +-----------------------------|--------+ >>> + * | >>> + * | >>> + * +------------------------| RPU if |----+ >>> + * | | | >>> + * | V | >>> + * | Realtime CPUs (RPU) | >>> + * | | >>> + * +--------------------------------------+ >>> + * | >>> + * +---------------------|----------------+ >>> + * | FPGA | | >>> + * | +-----------------------+ | >>> + * | | | | | >>> + * | +-------+ +-------+ +-------+ | >>> + * | | dev 1 | | dev 2 | | dev 3 | | >>> + * | +-------+ +-------+ +-------+ | >>> + * +--------------------------------------+ >>> + * >>> + * The RPU firmware extracts the device information from the loaded >> FPGA >>> + * image and implements a mechanism that allows the APU drivers to >>> + * enumerate such devices (device personality and resource details) via >>> + * a dedicated communication channel. >> >> What is APU? CPU resources that application uses? Then why cpu resources >> that DPDK >> Uses are not part of it? > > Hi Chenbo, > > APU's are application processor unit and are on-chip CPU's. So in short APUs > are CPUs only on which applications like DPDK are running. >
There are two sets of processing unit in the SoC. FW runs on RPU, everything else runs on APU (Arm cores), including OS, DPDK etc..
