> -----Original Message----- > From: Ferruh Yigit <ferruh.yi...@amd.com> > Sent: Tuesday, May 9, 2023 7:26 PM > To: Gupta, Nipun <nipun.gu...@amd.com>; Xia, Chenbo <chenbo....@intel.com>; > dev@dpdk.org; tho...@monjalon.net; david.march...@redhat.com > Cc: Anand, Harpreet <harpreet.an...@amd.com>; Agarwal, Nikhil > <nikhil.agar...@amd.com> > Subject: Re: [PATCH v4 1/4] bus/cdx: introduce cdx bus > > On 5/9/2023 12:09 PM, Gupta, Nipun wrote: > > [AMD Official Use Only - General] > > > > > > > >> -----Original Message----- > >> From: Xia, Chenbo <chenbo....@intel.com> > >> Sent: Tuesday, May 9, 2023 12:25 PM > >> To: Gupta, Nipun <nipun.gu...@amd.com>; dev@dpdk.org; > >> tho...@monjalon.net; david.march...@redhat.com > >> Cc: Yigit, Ferruh <ferruh.yi...@amd.com>; Anand, Harpreet > >> <harpreet.an...@amd.com>; Agarwal, Nikhil <nikhil.agar...@amd.com> > >> Subject: RE: [PATCH v4 1/4] bus/cdx: introduce cdx bus > >> > >> Caution: This message originated from an External Source. Use proper > >> caution when opening attachments, clicking links, or responding. > >> > >> > >>> -----Original Message----- > >>> From: Nipun Gupta <nipun.gu...@amd.com> > >>> Sent: Monday, May 8, 2023 7:18 PM > >>> To: dev@dpdk.org; tho...@monjalon.net; david.march...@redhat.com > >>> Cc: ferruh.yi...@amd.com; harpreet.an...@amd.com; > >> nikhil.agar...@amd.com; > >>> Nipun Gupta <nipun.gu...@amd.com> > >>> Subject: [PATCH v4 1/4] bus/cdx: introduce cdx bus > >>> > >>> CDX bus supports multiple type of devices, which can be > >>> exposed to user-space via vfio-cdx. > >>> > >>> vfio-cdx provides the MMIO IO_MEMORY regions as well as the > >>> DMA interface for the device (IOMMU). > >>> > >>> This support aims to enable the DPDK to support the cdx > >>> devices in user-space using VFIO interface. > >>> > >>> Signed-off-by: Nipun Gupta <nipun.gu...@amd.com> > >>> --- > >>> MAINTAINERS | 5 + > >>> doc/guides/rel_notes/release_23_07.rst | 6 + > >>> drivers/bus/cdx/bus_cdx_driver.h | 201 ++++++++++ > >>> drivers/bus/cdx/cdx.c | 520 > +++++++++++++++++++++++++ > >>> drivers/bus/cdx/cdx_logs.h | 37 ++ > >>> drivers/bus/cdx/cdx_vfio.c | 437 +++++++++++++++++++++ > >>> drivers/bus/cdx/meson.build | 13 + > >>> drivers/bus/cdx/private.h | 49 +++ > >>> drivers/bus/cdx/version.map | 11 + > >>> drivers/bus/meson.build | 1 + > >>> 10 files changed, 1280 insertions(+) > >>> create mode 100644 drivers/bus/cdx/bus_cdx_driver.h > >>> create mode 100644 drivers/bus/cdx/cdx.c > >>> create mode 100644 drivers/bus/cdx/cdx_logs.h > >>> create mode 100644 drivers/bus/cdx/cdx_vfio.c > >>> create mode 100644 drivers/bus/cdx/meson.build > >>> create mode 100644 drivers/bus/cdx/private.h > >>> create mode 100644 drivers/bus/cdx/version.map > >>> > >> > >> ... > >> > >>> --- /dev/null > >>> +++ b/drivers/bus/cdx/cdx.c > >>> @@ -0,0 +1,520 @@ > >>> +/* SPDX-License-Identifier: BSD-3-Clause > >>> + * Copyright (C) 2022-2023, Advanced Micro Devices, Inc. > >>> + */ > >>> + > >>> +/* > >>> + * Architecture Overview > >>> + * ===================== > >>> + * CDX is a Hardware Architecture designed for AMD FPGA devices. It > >>> + * consists of sophisticated mechanism for interaction between FPGA, > >>> + * Firmware and the APUs (Application CPUs). > >>> + * > >>> + * Firmware resides on RPU (Realtime CPUs) which interacts with > >>> + * the FPGA program manager and the APUs. The RPU provides memory- > >> mapped > >>> + * interface (RPU if) which is used to communicate with APUs. > >>> + * > >>> + * The diagram below shows an overview of the CDX architecture: > >>> + * > >>> + * +--------------------------------------+ > >>> + * | DPDK | > >>> + * | DPDK CDX drivers | > >>> + * | | | > >>> + * | DPDK CDX bus | > >>> + * | | | > >>> + * +-----------------------------|--------+ > >>> + * | > >>> + * +-----------------------------|--------+ > >>> + * | Application CPUs (APU) | | > >>> + * | | | > >>> + * | VFIO CDX driver | > >>> + * | Linux OS | | > >>> + * | Linux CDX bus | > >>> + * | | | > >>> + * +-----------------------------|--------+ > >>> + * | > >>> + * | > >>> + * +------------------------| RPU if |----+ > >>> + * | | | > >>> + * | V | > >>> + * | Realtime CPUs (RPU) | > >>> + * | | > >>> + * +--------------------------------------+ > >>> + * | > >>> + * +---------------------|----------------+ > >>> + * | FPGA | | > >>> + * | +-----------------------+ | > >>> + * | | | | | > >>> + * | +-------+ +-------+ +-------+ | > >>> + * | | dev 1 | | dev 2 | | dev 3 | | > >>> + * | +-------+ +-------+ +-------+ | > >>> + * +--------------------------------------+ > >>> + * > >>> + * The RPU firmware extracts the device information from the loaded > >> FPGA > >>> + * image and implements a mechanism that allows the APU drivers to > >>> + * enumerate such devices (device personality and resource details) > via > >>> + * a dedicated communication channel. > >> > >> What is APU? CPU resources that application uses? Then why cpu > resources > >> that DPDK > >> Uses are not part of it? > > > > Hi Chenbo, > > > > APU's are application processor unit and are on-chip CPU's. So in short > APUs > > are CPUs only on which applications like DPDK are running. > > > > There are two sets of processing unit in the SoC. FW runs on RPU, > everything else runs on APU (Arm cores), including OS, DPDK etc..
Cool, thanks Nipun and Ferruh for the explanation!
