RE: [PATCH 2/2] raw/ifpga/base: fix mmap retcode check fail
It looks good.
> -Original Message-
> From: Chengwen Feng
> Sent: Monday, February 6, 2023 18:53
> To: tho...@monjalon.net; ferruh.yi...@amd.com; Xu, Rosen
> ; Zhang, Tianfei ; Huang,
> Wei
> Cc: dev@dpdk.org
> Subject: [PATCH 2/2] raw/ifpga/base: fix mmap retcode check fail
>
> The MAP_FAILED should be used to determine whether the mapping is
> successful.
>
> Fixes: e41856b515ce ("raw/ifpga/base: enhance driver reliability in multi-
> process")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Chengwen Feng
> ---
> drivers/raw/ifpga/base/opae_hw_api.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/raw/ifpga/base/opae_hw_api.c
> b/drivers/raw/ifpga/base/opae_hw_api.c
> index 1117c3e160..6d48d227d6 100644
> --- a/drivers/raw/ifpga/base/opae_hw_api.c
> +++ b/drivers/raw/ifpga/base/opae_hw_api.c
> @@ -380,7 +380,7 @@ static pthread_mutex_t
> *opae_adapter_mutex_open(struct opae_adapter *adapter)
> PROT_READ | PROT_WRITE, MAP_SHARED,
> shm_id, 0);
> adapter->lock = (pthread_mutex_t *)ptr;
> - if (ptr) {
> + if (ptr != MAP_FAILED) {
> dev_info(NULL,
> "shared memory %s address is %p\n",
> shm_name, ptr);
> @@ -499,7 +499,7 @@ static void *opae_adapter_shm_alloc(struct
> opae_adapter *adapter)
> adapter->shm.size = size;
> adapter->shm.ptr = mmap(NULL, size, PROT_READ |
> PROT_WRITE,
> MAP_SHARED,
> shm_id, 0);
> - if (adapter->shm.ptr) {
> + if (adapter->shm.ptr != MAP_FAILED) {
> dev_info(NULL,
> "shared memory %s address is %p\n",
> shm_name, adapter->shm.ptr);
> --
> 2.17.1
Re: [dpdk-dev] [PATCH v2 1/1] raw/ifpga/base: check size before assigning
Hi Ferruh,
We have no way to run Coverity scan manually, if you or someone else know how
to do it, please let us know.
Thanks,
Wei
-Original Message-
From: Ferruh Yigit
Sent: Wednesday, April 14, 2021 15:42
To: Zhang, Tianfei ; Aaron Conole
Cc: David Marchand ; sta...@dpdk.org; Huang, Wei
; Zhang, Qi Z ; Xu, Rosen
; dev@dpdk.org; Mcnamara, John
Subject: Re: [PATCH v2 1/1] raw/ifpga/base: check size before assigning
On 4/14/2021 3:46 AM, Zhang, Tianfei wrote:
>
>
>> -Original Message-
>> From: Aaron Conole
>> Sent: 2021ๅนด4ๆ9ๆฅ 22:56
>> To: Yigit, Ferruh
>> Cc: David Marchand ; sta...@dpdk.org;
>> Zhang, Tianfei ; Huang, Wei
>> ; Zhang, Qi Z ; Xu, Rosen
>> ; dev@dpdk.org; Mcnamara, John
>>
>> Subject: Re: [PATCH v2 1/1] raw/ifpga/base: check size before
>> assigning
>>
>> Ferruh Yigit writes:
>>
>>> On 4/8/2021 9:51 AM, Wei Huang wrote:
>>>> In max10_staging_area_init(), variable "size" from fdt_get_reg()
>>>> may be invalid, it should be checked before assigning to member
>>>> variable "staging_area_size" of structure "intel_max10_device".
>>>>
>>>> Coverity issue: 367480, 367482
>>>> Fixes: 96ebfcf8125c ("raw/ifpga/base: add SPI and MAX10 device
>>>> driver")
>>>>
>>>> Signed-off-by: Wei Huang
>>>> ---
>>>> v2: check size before assigning to staging_area_size
>>>> ---
>>>>drivers/raw/ifpga/base/opae_intel_max10.c | 2 +-
>>>>drivers/raw/ifpga/base/opae_intel_max10.h | 1 +
>>>>2 files changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/drivers/raw/ifpga/base/opae_intel_max10.c
>>>> b/drivers/raw/ifpga/base/opae_intel_max10.c
>>>> index 443e248fb3..c223fafa03 100644
>>>> --- a/drivers/raw/ifpga/base/opae_intel_max10.c
>>>> +++ b/drivers/raw/ifpga/base/opae_intel_max10.c
>>>> @@ -593,7 +593,7 @@ static int max10_staging_area_init(struct
>> intel_max10_device *dev)
>>>>continue;
>>>> ret = fdt_get_reg(fdt_root, offset, 0, &start, &size); -if
>>>> (!ret) {
>>>> +if (!ret && (size <= MAX_STAGING_AREA_SIZE)) {
>>>>dev->staging_area_base = start;
>>>>dev->staging_area_size = size;
>>>>}
>>>> diff --git a/drivers/raw/ifpga/base/opae_intel_max10.h
>>>> b/drivers/raw/ifpga/base/opae_intel_max10.h
>>>> index 670683f017..e7142d6f0d 100644
>>>> --- a/drivers/raw/ifpga/base/opae_intel_max10.h
>>>> +++ b/drivers/raw/ifpga/base/opae_intel_max10.h
>>>> @@ -182,6 +182,7 @@ struct opae_retimer_status {
>>>>#define SBUS_VERSIONGENMASK(31, 16)
>>>> #define DFT_MAX_SIZE0x7e
>>>> +#define MAX_STAGING_AREA_SIZE0x380
>>>> int max10_reg_read(struct intel_max10_device *dev,
>>>>unsigned int reg, unsigned int *val);
>>>>
>>>
>>> Hi Aaron, David,
>>>
>>> The data flow is complex for this coverity issues [1], at least I
>>> can't confirm that change fixes the issue.
>>>
>>> Are you aware of any way to confirm this coverity issue before merging it?
>>
>> Not generically. :-/
>>
>> We need someone that understands the data flow and the coverity splat
>> to know that the fix is correct. Coverity even ratelimits how many
>> outstanding submissions we can post, iirc, so we don't get to push
>> patch sets (unless we pay? I don't recall if there's an option for that).
>
> This fix is looks good for me. The fdt_get_reg() function just read
> out the content of some items from DTS file, We call the libfdt library API
> to do this.
> The Coverity just assume some attacker broken the DTS file or invoke
> the function with arbitrary values, it is not safety, So this patch add some
> checking after the function return.
>
Hi Tianfei,
From the CI capacity perspective, it would be better to have a way to verify
Coverity fixes before merging them.
For most of the Coverity fixes we can tell that patch is fixing issue by
review, but for the non trivial cases like this, a way to verify patch, even it
is manual etc.., would be nice, otherwise it turns out to try and see Coverity
fixes.
And as far as I remember we had in the past that a Coverity fix result two more
new Coverity issues :)
>>
>>> [1]
>>> https://scan4.coverity.com/reports.htm#v26325/p10075/fileInstanceId=
>>> 10
>>> 0181086&defectInstanceId=14238477&mergedDefectId=367480
>
RE: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
> -Original Message-
> From: Stephen Hemminger
> Sent: Monday, June 6, 2022 23:38
> To: Huang, Wei
> Cc: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> hemant.agra...@nxp.com; sta...@dpdk.org; Xu, Rosen
> ; Zhang, Tianfei ; Zhang, Qi
> Z
> Subject: Re: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
>
> On Fri, 27 May 2022 01:37:00 -0400
> Wei Huang wrote:
>
> > +static inline int afu_mf_trylock(struct afu_mf_rawdev *dev) {
> > + int32_t x = 0;
> > +
> > + if (!dev || !dev->shared)
> > + return -ENODEV;
> > +
> > + x = __atomic_load_n(&dev->shared->lock, __ATOMIC_RELAXED);
> > +
> > + if ((x != 0) || (__atomic_compare_exchange_n(&dev->shared->lock,
> &x, 1,
> > + 1, __ATOMIC_ACQUIRE,
> __ATOMIC_RELAXED) == 0))
> > + return -EBUSY;
> > +
> > + return 0;
> > +}
> > +
> > +static inline void afu_mf_unlock(struct afu_mf_rawdev *dev) {
> > + if (!dev || !dev->shared)
> > + return;
> > +
> > + __atomic_store_n(&dev->shared->lock, 0, __ATOMIC_RELEASE); }
>
> What other subsystem is this shared with?
> Is there a good reason it can't just use existing spinlock?
This lock is used in multi-process situation, it prevent AFU from being
operated by two DPDK process simultaneously.
I can use spinlock to replace it, thanks.
RE: [PATCH v5 0/5] introduce afu_mf raw device driver
> -Original Message- > From: Zhang, Tianfei > Sent: Monday, June 6, 2022 09:48 > To: Huang, Wei ; dev@dpdk.org; > tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z > > Subject: RE: [PATCH v5 0/5] introduce afu_mf raw device driver > > > > > -----Original Message- > > From: Huang, Wei > > Sent: Friday, May 27, 2022 1:37 PM > > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com; > > hemant.agra...@nxp.com > > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei > > ; Zhang, Qi Z ; Huang, > > Wei > > Subject: [PATCH v5 0/5] introduce afu_mf raw device driver > > The title can change to: introduce AFU PMD driver of FPGA agree > > > > The first patch implements the framework of the AFU raw device driver. > > The first patch implements the framework of the AFU PMD driver based on > raw device interfaces. > > > The subsequent patches implement the driver of some AFUs. > > Can we add the guide about how to use or test those AFU PMD drivers in > documentation, like doc/guides/rawdevs/ifpga.rst. Add guide to file header. > > > > > Wei Huang (5): > > drivers/raw: introduce AFU raw device driver > > raw/afu_mf: add N3000 AFU driver > > raw/afu_mf: add HE-LBK AFU driver > > raw/afu_mf: add HE-MEM AFU driver > > raw/afu_mf: add HE-HSSI AFU driver > > > > drivers/raw/afu_mf/afu_mf_rawdev.c | 440 > > drivers/raw/afu_mf/afu_mf_rawdev.h | 89 ++ > > drivers/raw/afu_mf/he_hssi.c | 369 +++ > > drivers/raw/afu_mf/he_hssi.h | 102 ++ > > drivers/raw/afu_mf/he_lbk.c| 427 > > drivers/raw/afu_mf/he_lbk.h| 121 +++ > > drivers/raw/afu_mf/he_mem.c| 181 > > drivers/raw/afu_mf/he_mem.h| 40 + > > drivers/raw/afu_mf/meson.build |8 + > > drivers/raw/afu_mf/n3000_afu.c | 2005 > > > > drivers/raw/afu_mf/n3000_afu.h | 333 ++ > > drivers/raw/afu_mf/rte_pmd_afu.h | 134 +++ > > drivers/raw/afu_mf/version.map |3 + > > drivers/raw/meson.build|1 + > > 14 files changed, 4253 insertions(+) > > create mode 100644 drivers/raw/afu_mf/afu_mf_rawdev.c > > create mode 100644 drivers/raw/afu_mf/afu_mf_rawdev.h > > create mode 100644 drivers/raw/afu_mf/he_hssi.c create mode 100644 > > drivers/raw/afu_mf/he_hssi.h create mode 100644 > > drivers/raw/afu_mf/he_lbk.c create mode 100644 > > drivers/raw/afu_mf/he_lbk.h create mode 100644 > > drivers/raw/afu_mf/he_mem.c create mode 100644 > > drivers/raw/afu_mf/he_mem.h create mode 100644 > > drivers/raw/afu_mf/meson.build create mode 100644 > > drivers/raw/afu_mf/n3000_afu.c create mode 100644 > > drivers/raw/afu_mf/n3000_afu.h create mode 100644 > > drivers/raw/afu_mf/rte_pmd_afu.h create mode 100644 > > drivers/raw/afu_mf/version.map > > > > -- > > 1.8.3.1
RE: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
> -Original Message-
> From: Zhang, Tianfei
> Sent: Monday, June 6, 2022 09:53
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Subject: RE: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
>
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Friday, May 27, 2022 1:37 PM
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> > Wei
> > Subject: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
> >
> > Add multi-function AFU raw device driver to manage various AFU
> > (Acceleration Function Unit) in FPGA.
> > This driver implements common AFU raw device interfaces and exposes
> > them to application as standard raw device APIs.
> > Normal application can operate specified AFU as below, 1. call
> > rte_rawdev_pmd_get_named_dev() to find AFU raw device.
> > 2. call rte_rawdev_configure() to initialize AFU raw device.
> > 3. call rte_rawdev_selftest() to test function of AFU.
> >
> > Signed-off-by: Wei Huang
> > ---
> > v2: fix typo
> > ---
> > v3: fix build error in FreeBSD13-64, UB2004-32 and UB2204-32
> > ---
> > v4: fix coding style issue and build error in FreeBSD13-64
> > ---
> > v5: split patch into several patches
> > ---
> > drivers/raw/afu_mf/afu_mf_rawdev.c | 425
> > +
> > drivers/raw/afu_mf/afu_mf_rawdev.h | 71 +++
> > drivers/raw/afu_mf/meson.build | 5 +
> > drivers/raw/afu_mf/version.map | 3 +
> > drivers/raw/meson.build| 1 +
> > 5 files changed, 505 insertions(+)
> > create mode 100644 drivers/raw/afu_mf/afu_mf_rawdev.c
> > create mode 100644 drivers/raw/afu_mf/afu_mf_rawdev.h
> > create mode 100644 drivers/raw/afu_mf/meson.build create mode
> 100644
> > drivers/raw/afu_mf/version.map
>
> I am thinking that just put those afu* c/h files into drivers/raw/ifpga/
> folder
> is better? Because this AFU PMD driver is work on ifpga.
> And the file name of "afu_mf_rawdev.c" change to "afu_pmd_driver.c"?
>
agree
> >
> > diff --git a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > new file mode 100644
> > index 000..5be372a
> > --- /dev/null
> > +++ b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > @@ -0,0 +1,425 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright 2022 Intel Corporation
> > + */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include "afu_mf_rawdev.h"
> > +
> > +#define AFU_MF_PMD_RAWDEV_NAME rawdev_afu_mf
> > +
> > +static const struct rte_afu_uuid afu_uuid_map[] = {
> > + { 0, 0 /* sentinel */ }
> > +};
> > +
> > +static struct afu_mf_drv *afu_table[] = {
> > + NULL
> > +};
> > +
> > +static inline int afu_mf_trylock(struct afu_mf_rawdev *dev) {
> > + int32_t x = 0;
> > +
> > + if (!dev || !dev->shared)
> > + return -ENODEV;
> > +
> > + x = __atomic_load_n(&dev->shared->lock, __ATOMIC_RELAXED);
> > +
> > + if ((x != 0) || (__atomic_compare_exchange_n(&dev->shared->lock,
> &x,
> > 1,
> > + 1, __ATOMIC_ACQUIRE,
> __ATOMIC_RELAXED)
> > == 0))
> > + return -EBUSY;
> > +
> > + return 0;
> > +}
> > +
> > +static inline void afu_mf_unlock(struct afu_mf_rawdev *dev) {
> > + if (!dev || !dev->shared)
> > + return;
> > +
> > + __atomic_store_n(&dev->shared->lock, 0, __ATOMIC_RELEASE); }
> > +
> > +static int afu_mf_rawdev_configure(const struct rte_rawdev *rawdev,
> > + rte_rawdev_obj_t config, size_t config_size) {
> > + struct afu_mf_rawdev *dev = NULL;
> > + int ret = 0;
> > +
> > + AFU_MF_PMD_FUNC_TRACE();
> > +
> > + dev = afu_mf_rawdev_get_priv(rawdev);
> > + if (!dev)
> > + return -ENODEV;
> > +
> > + if (dev->ops && dev->ops->config)
> > + ret = (*dev->ops->config)(dev, config,
RE: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
> -Original Message-
> From: Zhang, Tianfei
> Sent: Monday, June 6, 2022 10:00
> To: Zhang, Tianfei ; Huang, Wei
> ; dev@dpdk.org; tho...@monjalon.net;
> nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Subject: RE: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
>
>
>
> > -Original Message-
> > From: Zhang, Tianfei
> > Sent: Monday, June 6, 2022 9:53 AM
> > To: Huang, Wei ; dev@dpdk.org;
> > tho...@monjalon.net; nipun.gu...@nxp.com;
> hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
> >
> > Subject: RE: [PATCH v5 1/5] drivers/raw: introduce AFU raw device
> > driver
> >
> >
> >
> > > -Original Message-
> > > From: Huang, Wei
> > > Sent: Friday, May 27, 2022 1:37 PM
> > > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > > hemant.agra...@nxp.com
> > > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > > ; Zhang, Qi Z ;
> > > Huang, Wei
> > > Subject: [PATCH v5 1/5] drivers/raw: introduce AFU raw device driver
> > >
> > > Add multi-function AFU raw device driver to manage various AFU
> > > (Acceleration Function Unit) in FPGA.
> > > This driver implements common AFU raw device interfaces and exposes
> > > them to application as standard raw device APIs.
> > > Normal application can operate specified AFU as below, 1. call
> > > rte_rawdev_pmd_get_named_dev() to find AFU raw device.
> > > 2. call rte_rawdev_configure() to initialize AFU raw device.
> > > 3. call rte_rawdev_selftest() to test function of AFU.
> > >
> > > Signed-off-by: Wei Huang
> > > ---
> > > v2: fix typo
> > > ---
> > > v3: fix build error in FreeBSD13-64, UB2004-32 and UB2204-32
> > > ---
> > > v4: fix coding style issue and build error in FreeBSD13-64
> > > ---
> > > v5: split patch into several patches
> > > ---
> > > drivers/raw/afu_mf/afu_mf_rawdev.c | 425
> > > +
> > > drivers/raw/afu_mf/afu_mf_rawdev.h | 71 +++
> > > drivers/raw/afu_mf/meson.build | 5 +
> > > drivers/raw/afu_mf/version.map | 3 +
> > > drivers/raw/meson.build| 1 +
> > > 5 files changed, 505 insertions(+)
> > > create mode 100644 drivers/raw/afu_mf/afu_mf_rawdev.c
> > > create mode 100644 drivers/raw/afu_mf/afu_mf_rawdev.h
> > > create mode 100644 drivers/raw/afu_mf/meson.build create mode
> > > 100644 drivers/raw/afu_mf/version.map
> >
> > I am thinking that just put those afu* c/h files into
> > drivers/raw/ifpga/ folder is better? Because this AFU PMD driver is work on
> ifpga.
> > And the file name of "afu_mf_rawdev.c" change to "afu_pmd_driver.c"?
>
> Ohm, the file name of "afu_mf_rawdev.c" change to "afu_pmd_core.c"?
> Because this file is the common code of AFU PMD driver, and specific
> function are in separate c file, like afu_pmd_n3000.c
>
agree
> >
> > >
> > > diff --git a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > > b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > > new file mode 100644
> > > index 000..5be372a
> > > --- /dev/null
> > > +++ b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > > @@ -0,0 +1,425 @@
> > > +/* SPDX-License-Identifier: BSD-3-Clause
> > > + * Copyright 2022 Intel Corporation */
> > > +
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +
> > > +#include
> > > +#include
> > > +#include
> > > +#include
> > > +
> > > +#include "afu_mf_rawdev.h"
> > > +
> > > +#define AFU_MF_PMD_RAWDEV_NAME rawdev_afu_mf
> > > +
> > > +static const struct rte_afu_uuid afu_uuid_map[] = {
> > > + { 0, 0 /* sentinel */ }
> > > +};
> > > +
> > > +static struct afu_mf_drv *afu_table[] = {
> > > + NULL
> > > +};
> > > +
> > > +static inline int afu_mf_trylock(struct afu_mf_rawdev *dev) {
> > > + int32_t x = 0;
> > > +
> > > + if (!dev || !dev->shared)
> > > + return -ENODEV;
> > > +
> > > + x = __atomic_load_n(&dev->shared->lock, __ATOMIC_RELAXED);
> > > +
> > >
RE: [PATCH v5 2/5] raw/afu_mf: add N3000 AFU driver
> -Original Message-
> From: Zhang, Tianfei
> Sent: Monday, June 6, 2022 09:39
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Subject: RE: [PATCH v5 2/5] raw/afu_mf: add N3000 AFU driver
>
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Friday, May 27, 2022 1:37 PM
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> Wei
> >
> > Subject: [PATCH v5 2/5] raw/afu_mf: add N3000 AFU driver
> >
> > N3000 AFU includes NLB0 and DMA modules, NLB0 is used to test PCI bus
> > and DMA is used to test local memory.
> > This driver initialize the modules and report test result.
> >
> > Signed-off-by: Wei Huang
> > ---
> > drivers/raw/afu_mf/afu_mf_rawdev.c |4 +
> > drivers/raw/afu_mf/afu_mf_rawdev.h | 18 +
> > drivers/raw/afu_mf/meson.build |4 +-
> > drivers/raw/afu_mf/n3000_afu.c | 2005
> >
> > drivers/raw/afu_mf/n3000_afu.h | 333 ++
> > drivers/raw/afu_mf/rte_pmd_afu.h | 97 ++
> > 6 files changed, 2460 insertions(+), 1 deletion(-)
> > create mode 100644 drivers/raw/afu_mf/n3000_afu.c
> > create mode 100644 drivers/raw/afu_mf/n3000_afu.h
> > create mode 100644 drivers/raw/afu_mf/rte_pmd_afu.h
> >
> > diff --git a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > index 5be372a..7c18f3b 100644
> > --- a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > +++ b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > @@ -17,15 +17,19 @@
> > #include
> > #include
> >
> > +#include "rte_pmd_afu.h"
> > #include "afu_mf_rawdev.h"
> > +#include "n3000_afu.h"
> >
> > #define AFU_MF_PMD_RAWDEV_NAME rawdev_afu_mf
> >
> > static const struct rte_afu_uuid afu_uuid_map[] = {
> > + { N3000_AFU_UUID_L, N3000_AFU_UUID_H },
> > { 0, 0 /* sentinel */ }
> > };
> >
> > static struct afu_mf_drv *afu_table[] = {
> > + &n3000_afu_drv,
> > NULL
> > };
> >
> > diff --git a/drivers/raw/afu_mf/afu_mf_rawdev.h
> > b/drivers/raw/afu_mf/afu_mf_rawdev.h
> > index df6715c..5a66f6c 100644
> > --- a/drivers/raw/afu_mf/afu_mf_rawdev.h
> > +++ b/drivers/raw/afu_mf/afu_mf_rawdev.h
> > @@ -30,6 +30,24 @@
> > #define AFU_MF_PMD_WARN(fmt, args...) \
> > AFU_MF_PMD_LOG(WARNING, fmt, ## args)
> >
> > +#define CLS_TO_SIZE(n) ((n) << 6) /* get size of n cache lines */
> > +#define SIZE_TO_CLS(s) ((s) >> 6) /* convert size to number of cache
> lines */
> > +#define MHZ(f) ((f) * 100)
> > +
> > +#define dsm_poll_timeout(addr, val, cond, invl, timeout) \
> > +({ \
> > + uint64_t __wait = 0; \
> > + uint64_t __invl = (invl);\
> > + uint64_t __timeout = (timeout); \
> > + for (; __wait <= __timeout; __wait += __invl) { \
> > + (val) = *(addr); \
> > + if (cond)\
> > + break; \
> > + rte_delay_ms(__invl);\
> > + }\
> > + (cond) ? 0 : 1; \
> > +})
>
> Dsm means DMA?
DSM means 'DMA Status Memory'
>
> > +
> > struct afu_mf_rawdev;
> >
> > struct afu_mf_ops {
> > diff --git a/drivers/raw/afu_mf/meson.build
> b/drivers/raw/afu_mf/meson.build
> > index 80526a2..8a989e3 100644
> > --- a/drivers/raw/afu_mf/meson.build
> > +++ b/drivers/raw/afu_mf/meson.build
> > @@ -2,4 +2,6 @@
> > # Copyright 2022 Intel Corporation
> >
> > deps += ['rawdev', 'bus_pci', 'bus_ifpga']
> > -sources = files('afu_mf_rawdev.c')
> > +sources = files('afu_mf_rawdev.c', 'n3000_afu.c')
> > +
> > +headers = files('rte_pmd_afu.h')
> > diff --git a/drivers/raw/afu_mf/n3000_afu.c
> b/drivers/raw/afu_mf/n3000_afu.c
> > new file mode 100644
> > index 000..19d7c54
> > --- /dev/null
> > +++ b/drivers/raw/afu_mf/n3000_afu.c
> > @@ -0,
RE: [PATCH v5 3/5] raw/afu_mf: add HE-LBK AFU driver
> -Original Message-
> From: Zhang, Tianfei
> Sent: Monday, June 6, 2022 09:41
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Subject: RE: [PATCH v5 3/5] raw/afu_mf: add HE-LBK AFU driver
>
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Friday, May 27, 2022 1:37 PM
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> > Wei
> > Subject: [PATCH v5 3/5] raw/afu_mf: add HE-LBK AFU driver
> >
> > HE-LBK and HE-MEM-LBK are host exerciser modules in OFS FPGA, HE-LBK
> > is used to test PCI bus and HE-MEM-LBK is used to test local memory.
> > This driver initialize the modules and report test result.
> >
> > Signed-off-by: Wei Huang
> > ---
> > drivers/raw/afu_mf/afu_mf_rawdev.c | 5 +
> > drivers/raw/afu_mf/he_lbk.c| 427
> > +
> > drivers/raw/afu_mf/he_lbk.h| 121 +++
> > drivers/raw/afu_mf/meson.build | 2 +-
> > drivers/raw/afu_mf/rte_pmd_afu.h | 14 ++
> > 5 files changed, 568 insertions(+), 1 deletion(-) create mode 100644
> > drivers/raw/afu_mf/he_lbk.c create mode 100644
> > drivers/raw/afu_mf/he_lbk.h
> >
> > diff --git a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > index 7c18f3b..e91eb21 100644
> > --- a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > +++ b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > @@ -20,16 +20,21 @@
> > #include "rte_pmd_afu.h"
> > #include "afu_mf_rawdev.h"
> > #include "n3000_afu.h"
> > +#include "he_lbk.h"
> >
> > #define AFU_MF_PMD_RAWDEV_NAME rawdev_afu_mf
> >
> > static const struct rte_afu_uuid afu_uuid_map[] = {
> > { N3000_AFU_UUID_L, N3000_AFU_UUID_H },
> > + { HE_LBK_UUID_L, HE_LBK_UUID_H },
> > + { HE_MEM_LBK_UUID_L, HE_MEM_LBK_UUID_H },
> > { 0, 0 /* sentinel */ }
> > };
> >
> > static struct afu_mf_drv *afu_table[] = {
> > &n3000_afu_drv,
> > + &he_lbk_drv,
> > + &he_mem_lbk_drv,
> > NULL
> > };
> >
> > diff --git a/drivers/raw/afu_mf/he_lbk.c b/drivers/raw/afu_mf/he_lbk.c
> > new file mode 100644 index 000..8735647
> > --- /dev/null
> > +++ b/drivers/raw/afu_mf/he_lbk.c
> > @@ -0,0 +1,427 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2022 Intel Corporation */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include "afu_mf_rawdev.h"
> > +#include "he_lbk.h"
> > +
> > +static int he_lbk_afu_config(struct afu_mf_rawdev *dev) {
> > + struct he_lbk_priv *priv = NULL;
> > + struct rte_pmd_afu_he_lbk_cfg *cfg = NULL;
> > + struct he_lbk_csr_cfg v;
> > +
> > + if (!dev)
> > + return -EINVAL;
> > +
> > + priv = (struct he_lbk_priv *)dev->priv;
> > + if (!priv)
> > + return -ENOENT;
> > +
> > + cfg = &priv->he_lbk_cfg;
> > +
> > + v.csr = 0;
> > +
> > + if (cfg->cont)
> > + v.cont = 1;
> > +
> > + v.mode = cfg->mode;
> > + v.trput_interleave = cfg->trput_interleave;
> > + if (cfg->multi_cl == 4)
> > + v.multicl_len = 2;
> > + else
> > + v.multicl_len = cfg->multi_cl - 1;
> > +
> > + AFU_MF_PMD_DEBUG("cfg: 0x%08x", v.csr);
> > + rte_write32(v.csr, priv->he_lbk_ctx.addr + CSR_CFG);
> > +
> > + return 0;
> > +}
> > +
> > +static void he_lbk_report(struct afu_mf_rawdev *dev, uint32_t cl) {
> > + struct he_lbk_priv *priv = NULL;
> > + struct rte_pmd_afu_he_lbk_cfg *cfg = NULL;
> > + struct he_lbk_ctx *ctx = NULL;
> > + struct he_lbk_dsm_status *stat = NULL;
> > + struct he_lbk_status0 stat0;
> > + struct he_lbk_status1 stat1;
> > + uint64_t swtest_msg = 0;
> > + uint64_t ticks = 0;
> > + uint64_t info = 0;
> >
RE: [PATCH v5 4/5] raw/afu_mf: add HE-MEM AFU driver
> -Original Message-
> From: Zhang, Tianfei
> Sent: Monday, June 6, 2022 14:37
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Subject: RE: [PATCH v5 4/5] raw/afu_mf: add HE-MEM AFU driver
>
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Friday, May 27, 2022 1:37 PM
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> > Wei
> > Subject: [PATCH v5 4/5] raw/afu_mf: add HE-MEM AFU driver
> >
> > HE-MEM is one of the host exerciser modules in OFS FPGA, which is used
> > to test local memory with built-in traffic generator.
> > This driver initialize the module and report test result.
> >
> > Signed-off-by: Wei Huang
> > ---
> > drivers/raw/afu_mf/afu_mf_rawdev.c | 3 +
> > drivers/raw/afu_mf/he_mem.c| 181
> > +
> > drivers/raw/afu_mf/he_mem.h| 40
> > drivers/raw/afu_mf/meson.build | 2 +-
> > drivers/raw/afu_mf/rte_pmd_afu.h | 7 ++
> > 5 files changed, 232 insertions(+), 1 deletion(-) create mode 100644
> > drivers/raw/afu_mf/he_mem.c create mode 100644
> > drivers/raw/afu_mf/he_mem.h
> >
> > diff --git a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > index e91eb21..a56f60e 100644
> > --- a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > +++ b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > @@ -21,6 +21,7 @@
> > #include "afu_mf_rawdev.h"
> > #include "n3000_afu.h"
> > #include "he_lbk.h"
> > +#include "he_mem.h"
> >
> > #define AFU_MF_PMD_RAWDEV_NAME rawdev_afu_mf
> >
> > @@ -28,6 +29,7 @@
> > { N3000_AFU_UUID_L, N3000_AFU_UUID_H },
> > { HE_LBK_UUID_L, HE_LBK_UUID_H },
> > { HE_MEM_LBK_UUID_L, HE_MEM_LBK_UUID_H },
> > + { HE_MEM_TG_UUID_L, HE_MEM_TG_UUID_H },
> > { 0, 0 /* sentinel */ }
> > };
> >
> > @@ -35,6 +37,7 @@
> > &n3000_afu_drv,
> > &he_lbk_drv,
> > &he_mem_lbk_drv,
> > + &he_mem_tg_drv,
> > NULL
> > };
> >
> > diff --git a/drivers/raw/afu_mf/he_mem.c
> b/drivers/raw/afu_mf/he_mem.c
> > new file mode 100644 index 000..ccbb3a8
> > --- /dev/null
> > +++ b/drivers/raw/afu_mf/he_mem.c
> > @@ -0,0 +1,181 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2022 Intel Corporation */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include "afu_mf_rawdev.h"
> > +#include "he_mem.h"
> > +
> > +static int he_mem_tg_test(struct afu_mf_rawdev *dev) {
> > + struct he_mem_tg_priv *priv = NULL;
> > + struct rte_pmd_afu_he_mem_tg_cfg *cfg = NULL;
> > + struct he_mem_tg_ctx *ctx = NULL;
> > + uint64_t value = 0x12345678;
> > + uint64_t cap = 0;
> > + uint64_t channel_mask = 0;
> > + int i, t = 0;
> > +
> > + if (!dev)
> > + return -EINVAL;
> > +
> > + priv = (struct he_mem_tg_priv *)dev->priv;
> > + if (!priv)
> > + return -ENOENT;
> > +
> > + cfg = &priv->he_mem_tg_cfg;
> > + ctx = &priv->he_mem_tg_ctx;
> > +
> > + AFU_MF_PMD_DEBUG("Channel mask: 0x%x", cfg->channel_mask);
> > +
> > + rte_write64(value, ctx->addr + MEM_TG_SCRATCHPAD);
> > + cap = rte_read64(ctx->addr + MEM_TG_SCRATCHPAD);
> > + AFU_MF_PMD_DEBUG("Scratchpad value: 0x%"PRIx64, cap);
> > + if (cap != value) {
> > + AFU_MF_PMD_ERR("Test scratchpad register failed");
> > + return -EIO;
> > + }
> > +
> > + cap = rte_read64(ctx->addr + MEM_TG_CTRL);
> > + AFU_MF_PMD_DEBUG("Capability: 0x%"PRIx64, cap);
> > +
> > + channel_mask = cfg->channel_mask & cap;
> > + /* start traffic generators */
> > + rte_write64(channel_mask, ctx->addr + MEM_TG_CTRL);
> > +
> > + /* check test status */
> > + while (t < MEM_TG_TIMEOU
RE: [PATCH v5 5/5] raw/afu_mf: add HE-HSSI AFU driver
> -Original Message-
> From: Zhang, Tianfei
> Sent: Monday, June 6, 2022 14:40
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Subject: RE: [PATCH v5 5/5] raw/afu_mf: add HE-HSSI AFU driver
>
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Friday, May 27, 2022 1:37 PM
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> > Wei
> > Subject: [PATCH v5 5/5] raw/afu_mf: add HE-HSSI AFU driver
> >
> > HE-HSSI is one of the host exerciser modules in OFS FPGA, which is
> > used to test HSSI (High Speed Serial Interface).
> > This driver initialize the module and report test result.
> >
> > Signed-off-by: Wei Huang
> > ---
> > drivers/raw/afu_mf/afu_mf_rawdev.c | 3 +
> > drivers/raw/afu_mf/he_hssi.c | 369
>
> The filename change to "afu_pmd_he_hssi.c" is better?
>
agree
> > +
> > drivers/raw/afu_mf/he_hssi.h | 102 ++
> > drivers/raw/afu_mf/meson.build | 3 +-
> > drivers/raw/afu_mf/rte_pmd_afu.h | 16 ++
> > 5 files changed, 492 insertions(+), 1 deletion(-) create mode 100644
> > drivers/raw/afu_mf/he_hssi.c create mode 100644
> > drivers/raw/afu_mf/he_hssi.h
> >
> > diff --git a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > index a56f60e..f24c748 100644
> > --- a/drivers/raw/afu_mf/afu_mf_rawdev.c
> > +++ b/drivers/raw/afu_mf/afu_mf_rawdev.c
> > @@ -22,6 +22,7 @@
> > #include "n3000_afu.h"
> > #include "he_lbk.h"
> > #include "he_mem.h"
> > +#include "he_hssi.h"
> >
> > #define AFU_MF_PMD_RAWDEV_NAME rawdev_afu_mf
> >
> > @@ -30,6 +31,7 @@
> > { HE_LBK_UUID_L, HE_LBK_UUID_H },
> > { HE_MEM_LBK_UUID_L, HE_MEM_LBK_UUID_H },
> > { HE_MEM_TG_UUID_L, HE_MEM_TG_UUID_H },
> > + { HE_HSSI_UUID_L, HE_HSSI_UUID_H },
> > { 0, 0 /* sentinel */ }
> > };
> >
> > @@ -38,6 +40,7 @@
> > &he_lbk_drv,
> > &he_mem_lbk_drv,
> > &he_mem_tg_drv,
> > + &he_hssi_drv,
> > NULL
> > };
> >
> > diff --git a/drivers/raw/afu_mf/he_hssi.c
> > b/drivers/raw/afu_mf/he_hssi.c new file mode 100644 index
> > 000..bedafbd
> > --- /dev/null
> > +++ b/drivers/raw/afu_mf/he_hssi.c
> > @@ -0,0 +1,369 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2022 Intel Corporation */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include "afu_mf_rawdev.h"
> > +#include "he_hssi.h"
> > +
> > +static int he_hssi_indirect_write(struct he_hssi_ctx *ctx, uint32_t addr,
> > + uint32_t value)
> > +{
> > + struct traffic_ctrl_cmd cmd;
> > + struct traffic_ctrl_data data;
> > + uint32_t i = 0;
> > +
> > + AFU_MF_PMD_DEBUG("Indirect write 0x%x, value 0x%08x", addr,
> > value);
> > +
> > + if (!ctx)
> > + return -EINVAL;
> > +
> > + data.write_data = value;
> > + rte_write64(data.csr, ctx->addr + TRAFFIC_CTRL_DATA);
> > +
> > + cmd.csr = 0;
> > + cmd.write_cmd = 1;
> > + cmd.afu_cmd_addr = addr;
> > + rte_write64(cmd.csr, ctx->addr + TRAFFIC_CTRL_CMD);
> > +
> > + while (i < MAILBOX_TIMEOUT_MS) {
> > + rte_delay_ms(MAILBOX_POLL_INTERVAL_MS);
> > + cmd.csr = rte_read64(ctx->addr + TRAFFIC_CTRL_CMD);
> > + if (cmd.ack_trans)
> > + break;
> > + i += MAILBOX_POLL_INTERVAL_MS;
> > + }
> > + if (i >= MAILBOX_TIMEOUT_MS)
> > + return -ETIMEDOUT;
> > +
> > + i = 0;
> > + cmd.csr = 0;
> > + while (i < MAILBOX_TIMEOUT_MS) {
> > + cmd.ack_trans = 1;
> > + rte_write64(cmd.csr, ctx->addr + TRAFFIC_CTRL_CMD);
> > + rte_delay_ms(MAILBOX_POLL_INTERVAL_MS);
> > +
RE: [PATCH v2 1/2] raw/ifpga/base: add pmci driver
> -Original Message-
> From: Zhang, Tianfei
> Sent: Monday, June 13, 2022 15:01
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Subject: RE: [PATCH v2 1/2] raw/ifpga/base: add pmci driver
>
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Friday, June 10, 2022 10:25 AM
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> Wei
> >
> > Subject: [PATCH v2 1/2] raw/ifpga/base: add pmci driver
> >
> > PMCI (Platform Management Control Interface) is a new module in FPGA,
> > which is designed to cooperate with BMC chip to fulfill board management
> > functions.
> > This driver provide interfaces to access registers of BMC chip and
> > external flash of FPGA.
> >
> > Signed-off-by: Wei Huang
> > ---
> > drivers/raw/ifpga/base/ifpga_defines.h | 103 ++-
> > drivers/raw/ifpga/base/ifpga_feature_dev.c | 2 +
> > drivers/raw/ifpga/base/ifpga_feature_dev.h | 1 +
> > drivers/raw/ifpga/base/ifpga_fme.c | 265 +--
> > drivers/raw/ifpga/base/ifpga_fme_error.c | 2 +
> > drivers/raw/ifpga/base/ifpga_port_error.c| 2 +-
> > drivers/raw/ifpga/base/ifpga_sec_mgr.c | 9 +-
> > drivers/raw/ifpga/base/ifpga_sec_mgr.h | 9 +-
> > drivers/raw/ifpga/base/opae_hw_api.c | 29 +
> > drivers/raw/ifpga/base/opae_hw_api.h | 1 +
> > drivers/raw/ifpga/base/opae_intel_max10.c| 938
> > ---
> > drivers/raw/ifpga/base/opae_intel_max10.h| 313 +++-
> > drivers/raw/ifpga/base/opae_osdep.h | 43 +-
> > drivers/raw/ifpga/base/osdep_rte/osdep_generic.h | 10 +
> > 14 files changed, 1524 insertions(+), 203 deletions(-)
>
> This patch is very huge. I thinks this patch can slit into 3 patches: base
> PMCI
> driver, flash support and sensor support.
>
Agree, let me split it into three patches.
> >
> > diff --git a/drivers/raw/ifpga/base/ifpga_defines.h
> > b/drivers/raw/ifpga/base/ifpga_defines.h
> > index 9a280eb..7c8fa89 100644
> > --- a/drivers/raw/ifpga/base/ifpga_defines.h
> > +++ b/drivers/raw/ifpga/base/ifpga_defines.h
> > @@ -23,6 +23,7 @@
> > #define FME_FEATURE_NIOS_SPI"fme_nios_spi"
> > #define FME_FEATURE_I2C_MASTER "fme_i2c_master"
> > #define FME_FEATURE_ETH_GROUP "fme_eth_group"
> > +#define FME_FEATURE_PMCI"fme_pmci"
> >
> > #define PORT_FEATURE_HEADER "port_hdr"
> > #define PORT_FEATURE_UAFU "port_uafu"
> > @@ -91,6 +92,7 @@ enum fpga_id_type {
> > #define FME_FEATURE_ID_NIOS_SPI 0xd
> > #define FME_FEATURE_ID_I2C_MASTER 0xf
> > #define FME_FEATURE_ID_ETH_GROUP 0x10
> > +#define FME_FEATURE_ID_PMCI 0x12
> >
> > #define PORT_FEATURE_ID_HEADER FEATURE_ID_FIU_HEADER
> > #define PORT_FEATURE_ID_ERROR 0x10
> > @@ -266,6 +268,24 @@ struct feature_fme_bitstream_id {
> > union {
> > u64 csr;
> > struct {
> > + u8 build_patch:8;
> > + u8 build_minor:8;
> > + u8 build_major:8;
> > + u8 fvl_bypass:1;
> > + u8 mac_lightweight:1;
> > + u8 disagregate:1;
> > + u8 lightweiht:1;
> > + u8 seu:1;
> > + u8 ptp:1;
> > + u8 reserve:2;
> > + u8 interface:4;
> > + u32 afu_revision:12;
> > + u8 patch:4;
> > + u8 minor:4;
> > + u8 major:4;
> > + u8 reserved:4;
> > + } v1;
> > + struct {
> > u32 gitrepo_hash:32;/* GIT repository hash */
> > /*
> > * HSSI configuration identifier:
> > @@ -274,7 +294,8 @@ struct feature_fme_bitstream_id {
> > * 2 - Ethernet
> > */
> > u8 hssi_id:4;
> > - u16 rsvd1:12; /* Reserved */
> > + u8 rsvd1:4;
> > + u8 fim_type:8;
> > /* Bitstream version patch number *
RE: [PATCH v7 1/5] raw/ifpga: introduce AFU driver framework
> -Original Message-
> From: Xu, Rosen
> Sent: Wednesday, June 15, 2022 14:55
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Zhang, Qi Z
>
> Subject: RE: [PATCH v7 1/5] raw/ifpga: introduce AFU driver framework
>
> Hi Wei,
>
> Some comments.
>
> Thanks,
> Rosen
>
> > -Original Message-
> > From: Huang, Wei
> > Sent: Thursday, June 09, 2022 15:37
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> > Wei
> > Subject: [PATCH v7 1/5] raw/ifpga: introduce AFU driver framework
> >
> > AFU (Acceleration Function Unit) is part of FPGA and enumerated by
> > ifpga driver.
> > This driver implements common AFU device interfaces and exposes them
> > to application as standard raw device APIs.
> > Normally application can operate specified AFU as below, 1. call
> > rte_rawdev_pmd_get_named_dev() to find AFU device.
> > 2. call rte_rawdev_configure() to initialize AFU device.
> > 3. call rte_rawdev_selftest() to test AFU device.
> >
> > Signed-off-by: Wei Huang
> > ---
> > v2: fix typo
> > ---
> > v3: fix build error in FreeBSD13-64, UB2004-32 and UB2204-32
> > ---
> > v4: fix coding style issue and build error in FreeBSD13-64
> > ---
> > v5: split patch into several patches
> > ---
> > v6: move source files to ifpga and rename, use spinlock
> > ---
> > drivers/raw/ifpga/afu_pmd_core.c | 453
> > +++
> > drivers/raw/ifpga/afu_pmd_core.h | 76 +++
> > drivers/raw/ifpga/meson.build| 2 +-
> > 3 files changed, 530 insertions(+), 1 deletion(-) create mode 100644
> > drivers/raw/ifpga/afu_pmd_core.c create mode 100644
> > drivers/raw/ifpga/afu_pmd_core.h
> >
> > diff --git a/drivers/raw/ifpga/afu_pmd_core.c
> > b/drivers/raw/ifpga/afu_pmd_core.c
> > new file mode 100644
> > index 000..a24b517
> > --- /dev/null
> > +++ b/drivers/raw/ifpga/afu_pmd_core.c
> > @@ -0,0 +1,453 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright 2022 Intel Corporation
> > + *
> > + * AFU is Acceleration Function Unit in FPGA, it is enumerated by ifpga
> pmd.
> > + * Suppose AFU is found in FPGA at PCI address 31:00.0, then you can
> > +create
> > + * and test a AFU device by following steps in application.
> > + * 1. rte_vdev_init("ifpga_rawdev_cfg0", "ifpga=31:00.0,port=0")
> > + * 2. rawdev = rte_rawdev_pmd_get_named_dev("afu_0|31:00.0")
> > + * 3. rte_rawdev_configure(rawdev->dev_id, &cfg, sizeof(cfg))
> > + * 4. rte_rawdev_selftest(rawdev->dev_id)
> > + * 5. rte_vdev_uninit("ifpga_rawdev_cfg0")
> > + *
> > + * AFU device name format used in rte_rawdev_pmd_get_named_dev is
> > + * "afu_[port]|[BDF]". Please refer to OPAE documentation for the
> > +meaning of
> > + * port. Each AFU device has specific configuration data, they are
> > +defined
> > + * in rte_pmd_afu.h.
> > + *
> > + */
> > +
> Could you split these notes? General speaking we just announce license in
> head of source file.
>
Agree
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include "afu_pmd_core.h"
> > +
> > +static struct rte_afu_uuid
> > afu_pmd_uuid_map[AFU_RAWDEV_MAX_DRVS+1];
> > +TAILQ_HEAD(afu_drv_list, afu_rawdev_drv); static struct afu_drv_list
> > +afu_pmd_list = TAILQ_HEAD_INITIALIZER(afu_pmd_list);
> > +
> > +static inline int afu_rawdev_trylock(struct afu_rawdev *dev) {
> > + if (!dev || !dev->sd)
> > + return 0;
> > +
> > + return rte_spinlock_trylock(&dev->sd->lock);
> > +}
> > +
> > +static inline void afu_rawdev_unlock(struct afu_rawdev *dev) {
> > + if (!dev || !dev->sd)
> > + return;
> > +
> > + rte_spinlock_unlock(&dev->sd->lock);
> > +}
> > +
> > +static int afu_rawdev_configure(const struct rte_rawdev *rawdev,
> > + rte_rawdev_obj_t config, size_t config_size) {
> > + struct afu_rawdev *dev = NULL;
> > + int ret = 0;
> > +
> > + IFPGA_RAWDE
RE: [PATCH v7 2/5] raw/ifpga: add N3000 AFU driver
> -Original Message-
> From: Xu, Rosen
> Sent: Wednesday, June 15, 2022 15:08
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Zhang, Qi Z
>
> Subject: RE: [PATCH v7 2/5] raw/ifpga: add N3000 AFU driver
>
> Hi Wei,
>
> Some comments.
>
> Thanks,
> Rosen
>
> > -Original Message-
> > From: Huang, Wei
> > Sent: Thursday, June 09, 2022 15:37
> > To: dev@dpdk.org; tho...@monjalon.net; nipun.gu...@nxp.com;
> > hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei
> > ; Zhang, Qi Z ; Huang,
> Wei
> >
> > Subject: [PATCH v7 2/5] raw/ifpga: add N3000 AFU driver
> >
> > N3000 AFU includes NLB0 and DMA modules, NLB0 is used to test PCI bus
> > and DMA is used to test local memory.
> > This driver initialize the modules and report test result.
> >
> > Signed-off-by: Wei Huang
> > ---
> > v2: move source files to ifpga and rename, refine code
> > ---
> > v3: fix Ubuntu 20.04 ARM build
> > ---
> > drivers/raw/ifpga/afu_pmd_core.h | 19 +
> > drivers/raw/ifpga/afu_pmd_n3000.c | 2019
> > +
> > drivers/raw/ifpga/afu_pmd_n3000.h | 339 +++
> > drivers/raw/ifpga/meson.build |3 +-
> > drivers/raw/ifpga/rte_pmd_afu.h | 97 ++
> > 5 files changed, 2476 insertions(+), 1 deletion(-)
> > create mode 100644 drivers/raw/ifpga/afu_pmd_n3000.c
> > create mode 100644 drivers/raw/ifpga/afu_pmd_n3000.h
> > create mode 100644 drivers/raw/ifpga/rte_pmd_afu.h
> >
> > diff --git a/drivers/raw/ifpga/afu_pmd_core.h
> > b/drivers/raw/ifpga/afu_pmd_core.h
> > index 4fad2c7..a938172 100644
> > --- a/drivers/raw/ifpga/afu_pmd_core.h
> > +++ b/drivers/raw/ifpga/afu_pmd_core.h
> > @@ -14,6 +14,7 @@
> > #include
> >
> > #include
> > +#include
> > #include
> > #include
> >
> > @@ -60,6 +61,24 @@ struct afu_rawdev {
> > return rawdev ? (struct afu_rawdev *)rawdev->dev_private : NULL;
> > }
> >
> > +#define CLS_TO_SIZE(n) ((n) << 6) /* get size of n cache lines */
> > +#define SIZE_TO_CLS(s) ((s) >> 6) /* convert size to number of cache
> lines
> > */
> > +#define MHZ(f) ((f) * 100)
> > +
> > +#define dsm_poll_timeout(addr, val, cond, invl, timeout) \
> > +({ \
> > + uint64_t __wait = 0; \
> > + uint64_t __invl = (invl);\
> > + uint64_t __timeout = (timeout); \
> > + for (; __wait <= __timeout; __wait += __invl) { \
> > + (val) = *(addr); \
> > + if (cond)\
> > + break; \
> > + rte_delay_ms(__invl);\
> > + }\
> > + (cond) ? 0 : 1; \
> > +})
> > +
> > void afu_pmd_register(struct afu_rawdev_drv *driver);
> > void afu_pmd_unregister(struct afu_rawdev_drv *driver);
> >
> > diff --git a/drivers/raw/ifpga/afu_pmd_n3000.c
> > b/drivers/raw/ifpga/afu_pmd_n3000.c
> > new file mode 100644
> > index 000..14f7fe0
> > --- /dev/null
> > +++ b/drivers/raw/ifpga/afu_pmd_n3000.c
> > @@ -0,0 +1,2019 @@
> > +/* SPDX-License-Identifier: BSD-3-Clause
> > + * Copyright(c) 2022 Intel Corporation
> > + */
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +#include
> > +
> > +#include "afu_pmd_core.h"
> > +#include "afu_pmd_n3000.h"
> > +
> > +static int nlb_afu_config(struct afu_rawdev *dev)
> > +{
> > + struct n3000_afu_priv *priv = NULL;
> > + struct rte_pmd_afu_nlb_cfg *cfg = NULL;
> > + struct nlb_csr_cfg v;
> > +
> > + if (!dev)
> > + return -EINVAL;
> > +
> > + if (!dev->priv)
> > + return -ENOENT;
> > +
> > + priv = (struct n3000_afu_priv *)dev->priv;
> > + cfg = &priv->nlb
RE: [PATCH] doc: announce some raw/ifpga API removal
Hi David, > -Original Message- > From: Xu, Rosen > Sent: Friday, July 1, 2022 14:16 > To: David Marchand ; dev@dpdk.org; Zhang, > Tianfei ; Huang, Wei > Cc: Ray Kinsella > Subject: RE: [PATCH] doc: announce some raw/ifpga API removal > > Hi David, > > Got, thank you so much. > > > > -Original Message- > > From: David Marchand > > Sent: Thursday, June 30, 2022 17:41 > > To: dev@dpdk.org; Xu, Rosen ; Zhang, Tianfei > > > > Cc: Ray Kinsella > > Subject: [PATCH] doc: announce some raw/ifpga API removal > > > > rte_pmd_ifpga_get_pci_bus() documentation is vague and it is unclear > > what could be done with it. > > On the other hand, EAL provides a standard API to retrieve a bus > > object by name. > > Agree, this API is used in an external application, I can use rte_get_bus_by_name() to replace it. I will submit a patch to remove rte_pmd_ifpga_get_pci_bus() after DPDK22.07 is released. > > Announce removal of this driver specific API for v22.11. > > > > Signed-off-by: David Marchand > > --- > > A RFC series of the intended changes is available at: > > > https://patches.dpdk.org/project/dpdk/list/?series=23811&state=%2A&arc > > h > > ive=both > > > > --- > > doc/guides/rel_notes/deprecation.rst | 2 ++ > > 1 file changed, 2 insertions(+) > > > > diff --git a/doc/guides/rel_notes/deprecation.rst > > b/doc/guides/rel_notes/deprecation.rst > > index 4e5b23c53d..64d649777a 100644 > > --- a/doc/guides/rel_notes/deprecation.rst > > +++ b/doc/guides/rel_notes/deprecation.rst > > @@ -119,6 +119,8 @@ Deprecation Notices > > * metrics: The function ``rte_metrics_init`` will have a non-void return > >in order to notify errors instead of calling ``rte_exit``. > > > > +* raw/ifgpa: The ``rte_pmd_ifpga_get_pci_bus`` will be removed in 22.11. > > + > > * raw/ioat: The ``ioat`` rawdev driver has been deprecated, since it's > >functionality is provided through the new ``dmadev`` infrastructure. > >To continue to use hardware previously supported by the ``ioat`` > > rawdev driver, > > -- > > 2.36.1
RE: [PATCH] doc: announce some raw/ifpga API removal
> -Original Message-
> From: David Marchand
> Sent: Friday, July 1, 2022 15:01
> To: Huang, Wei ; Xu, Rosen ;
> Zhang, Tianfei
> Cc: dev@dpdk.org; Ray Kinsella
> Subject: Re: [PATCH] doc: announce some raw/ifpga API removal
>
> On Fri, Jul 1, 2022 at 8:25 AM Huang, Wei wrote:
> > > >
> > > > rte_pmd_ifpga_get_pci_bus() documentation is vague and it is
> > > > unclear what could be done with it.
> > > > On the other hand, EAL provides a standard API to retrieve a bus
> > > > object by name.
> > > >
> > Agree, this API is used in an external application, I can use
> rte_get_bus_by_name() to replace it.
>
> What is the PCI bus used for, in this application?
>
In this application, target PCI device is Intel FPGA, it supports some special
operation like removing it from PCI bus and rescanning it back from PCI bus,
So there are two things need to be done directly on rte_pci_bus:
1. Rescan PCI bus
pci_bus->bus.scan()
2. Get pci_dev by specified PCI address, and remove it
TAILQ_FOREACH(pci_dev, &pci_bus->device_list, next) {
if (!rte_pci_addr_cmp(&pci_dev->addr, &addr))
return pci_dev;
}
pci_drv = pci_dev->driver;
pci_drv->remove(pci_dev);
>
> > I will submit a patch to remove rte_pmd_ifpga_get_pci_bus() after DPDK22.07
> is released.
>
> I sent a patch a few days ago:
> https://patches.dpdk.org/project/dpdk/patch/20220628144643.1213026-3-
> david.march...@redhat.com/
>
>
> --
> David Marchand
RE: [PATCH] doc: announce some raw/ifpga API removal
> -Original Message-
> From: David Marchand
> Sent: Friday, July 1, 2022 15:22
> To: Huang, Wei
> Cc: Xu, Rosen ; Zhang, Tianfei ;
> dev@dpdk.org; Ray Kinsella
> Subject: Re: [PATCH] doc: announce some raw/ifpga API removal
>
> On Fri, Jul 1, 2022 at 9:16 AM Huang, Wei wrote:
> > > What is the PCI bus used for, in this application?
> > >
> > In this application, target PCI device is Intel FPGA, it supports some
> > special operation like removing it from PCI bus and rescanning it back from
> > PCI
> bus, So there are two things need to be done directly on rte_pci_bus:
> > 1. Rescan PCI bus
> > pci_bus->bus.scan()
> > 2. Get pci_dev by specified PCI address, and remove it
> > TAILQ_FOREACH(pci_dev, &pci_bus->device_list, next) {
> > if (!rte_pci_addr_cmp(&pci_dev->addr, &addr))
> > return pci_dev;
> > }
> >
> > pci_drv = pci_dev->driver;
> > pci_drv->remove(pci_dev);
>
> Can't this application use rte_dev_remove and rte_dev_probe?
> If not, we should add the missing parts in the API.
>
Both rte_dev_remove and rte_dev_probe need rte_device pointer. In this
application, it only know the device's PCI address, is there an
existing API to get the rte_pci_device pointer by its PCI address ?
For PCI rescan, I know there is an API called rte_bus_scan(), which seems to be
able to replace pci_bus->bus.scan(). But is it reasonable to scan all buses
when I want to only scan PCI ?
Why I need to rescan PCI bus, the answer is current PCI scan only add PCI
device to rte_pci_bus when it is bound to kernel driver. In our case, a FPGA
may not bind to any driver, this application bind vfio-pci to it and call bus
scan to add it to rte_pci_bus.
>
> --
> David Marchand
RE: [PATCH] doc: announce some raw/ifpga API removal
> -Original Message-
> From: David Marchand
> Sent: Friday, July 1, 2022 16:15
> To: Huang, Wei
> Cc: Xu, Rosen ; Zhang, Tianfei ;
> dev@dpdk.org; Ray Kinsella
> Subject: Re: [PATCH] doc: announce some raw/ifpga API removal
>
> On Fri, Jul 1, 2022 at 10:09 AM David Marchand
> wrote:
> > > > > 2. Get pci_dev by specified PCI address, and remove it
> > > > > TAILQ_FOREACH(pci_dev, &pci_bus->device_list, next) {
> > > > > if (!rte_pci_addr_cmp(&pci_dev->addr, &addr))
> > > > > return pci_dev;
> > > > > }
> > > > >
> > > > > pci_drv = pci_dev->driver;
> > > > > pci_drv->remove(pci_dev);
> > > >
> > > > Can't this application use rte_dev_remove and rte_dev_probe?
> > > > If not, we should add the missing parts in the API.
> > > >
> > > Both rte_dev_remove and rte_dev_probe need rte_device pointer. In
> > > this application, it only know the device's PCI address, is there an
> > > existing API
> to get the rte_pci_device pointer by its PCI address ?
> >
> > rte_dev_probe takes a devargs string as input.
> > int rte_dev_probe(const char *devargs);
> >
> > You need the rte_device for removing which can be retrieved from
> > rte_rawdev_info_get.
>
> Additionnaly, rte_eal_hotplug_{add,remove} do the same jobs, but with an
> easier(?) interface.
>
>
I checked rte_eal_hotplug_{add,remove}, they should meet my requirements,
thanks a lot.
> --
> David Marchand
RE: [PATCH] doc: announce some raw/ifpga API removal
> -Original Message-
> From: David Marchand
> Sent: Thursday, July 7, 2022 17:30
> To: Huang, Wei ; Zhang, Tianfei
> ; Xu, Rosen
> Cc: dev@dpdk.org; Ray Kinsella
> Subject: Re: [PATCH] doc: announce some raw/ifpga API removal
>
> On Fri, Jul 1, 2022 at 10:35 AM David Marchand
> wrote:
> > > > > > > Can't this application use rte_dev_remove and rte_dev_probe?
> > > > > > > If not, we should add the missing parts in the API.
> > > > > > >
> > > > > > Both rte_dev_remove and rte_dev_probe need rte_device pointer.
> > > > > > In this application, it only know the device's PCI address, is
> > > > > > there an existing API
> > > > to get the rte_pci_device pointer by its PCI address ?
> > > > >
> > > > > rte_dev_probe takes a devargs string as input.
> > > > > int rte_dev_probe(const char *devargs);
> > > > >
> > > > > You need the rte_device for removing which can be retrieved from
> > > > > rte_rawdev_info_get.
> > > >
> > > > Additionnaly, rte_eal_hotplug_{add,remove} do the same jobs, but
> > > > with an
> > > > easier(?) interface.
> > > >
> > > >
> > > I checked rte_eal_hotplug_{add,remove}, they should meet my requirements,
> thanks a lot.
> >
> > Cool, thanks.
>
> So it seems there is nothing blocking for this API change.
> Could you ack the deprecation notice?
>
This API can be removed.
Acked
> Thanks.
>
> --
> David Marchand
RE: [PATCH v1] raw/ifpga: fix ifpga devices cleanup function
Hi,
> -Original Message-
> From: Yigit, Ferruh
> Sent: Wednesday, January 26, 2022 21:25
> To: Huang, Wei ; dev@dpdk.org; Xu, Rosen
> ; Zhang, Qi Z ; Nipun Gupta
> ; Hemant Agrawal
> Cc: sta...@dpdk.org; Zhang, Tianfei
> Subject: Re: [PATCH v1] raw/ifpga: fix ifpga devices cleanup function
>
> On 1/26/2022 3:29 AM, Wei Huang wrote:
> > Use rte_dev_remove() to replace rte_rawdev_pmd_release() in
> > ifpga_rawdev_cleanup(), resources occupied by ifpga raw devices such
> > as threads can be released correctly.
> >
>
> As far as I understand you are fixing an issue that not all resources are
> released, is this correct?
> What are these not released resources?
>
> And 'rte_rawdev_pmd_release()' rawdev API seems intended to do the
> cleanup, is it expected that some resources are not freed after this call, or
> should we fix that API?
> If the device remove API needs to be used, what is the point of
> 'rte_rawdev_pmd_release()' API?
>
> cc'ed rawdev maintainers for comment.
Yes, this patch is to release all the resources of ifpga_rawdev after testpmd
exit, the not released resources are interrupt and thread.
rte_rawdev_pmd_release implemented in ifpga_rawdev only release memory
allocated by ifpga driver, that's the expected behavior.
I think it's a simple and safe way to release resources completely by calling
rte_dev_remove.
>
> > Fixes: f724a802 ("raw/ifpga: add miscellaneous APIs")
> >
> > Signed-off-by: Wei Huang
> > ---
> > drivers/raw/ifpga/ifpga_rawdev.c | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/raw/ifpga/ifpga_rawdev.c
> > b/drivers/raw/ifpga/ifpga_rawdev.c
> > index fdf3c23..88c38aa 100644
> > --- a/drivers/raw/ifpga/ifpga_rawdev.c
> > +++ b/drivers/raw/ifpga/ifpga_rawdev.c
> > @@ -1787,12 +1787,14 @@ int ifpga_rawdev_partial_reconfigure(struct
> rte_rawdev *dev, int port,
> > void ifpga_rawdev_cleanup(void)
> > {
> > struct ifpga_rawdev *dev;
> > + struct rte_rawdev *rdev;
> > unsigned int i;
> >
> > for (i = 0; i < IFPGA_RAWDEV_NUM; i++) {
> > dev = &ifpga_rawdevices[i];
> > if (dev->rawdev) {
> > - rte_rawdev_pmd_release(dev->rawdev);
> > + rdev = dev->rawdev;
> > + rte_dev_remove(rdev->device);
> > dev->rawdev = NULL;
> > }
> > }
RE: [PATCH v1] raw/ifpga: fix ifpga devices cleanup function
Hi,
> -Original Message-
> From: Yigit, Ferruh
> Sent: Thursday, January 27, 2022 20:34
> To: Huang, Wei ; dev@dpdk.org; Xu, Rosen
> ; Zhang, Qi Z ; Nipun Gupta
> ; Hemant Agrawal
> Cc: sta...@dpdk.org; Zhang, Tianfei
> Subject: Re: [PATCH v1] raw/ifpga: fix ifpga devices cleanup function
>
> On 1/27/2022 8:57 AM, Huang, Wei wrote:
> > Hi,
> >
> >> -Original Message-
> >> From: Yigit, Ferruh
> >> Sent: Wednesday, January 26, 2022 21:25
> >> To: Huang, Wei ; dev@dpdk.org; Xu, Rosen
> >> ; Zhang, Qi Z ; Nipun Gupta
> >> ; Hemant Agrawal
> >> Cc: sta...@dpdk.org; Zhang, Tianfei
> >> Subject: Re: [PATCH v1] raw/ifpga: fix ifpga devices cleanup function
> >>
> >> On 1/26/2022 3:29 AM, Wei Huang wrote:
> >>> Use rte_dev_remove() to replace rte_rawdev_pmd_release() in
> >>> ifpga_rawdev_cleanup(), resources occupied by ifpga raw devices such
> >>> as threads can be released correctly.
> >>>
> >>
> >> As far as I understand you are fixing an issue that not all resources
> >> are released, is this correct?
> >> What are these not released resources?
> >>
> >> And 'rte_rawdev_pmd_release()' rawdev API seems intended to do the
> >> cleanup, is it expected that some resources are not freed after this
> >> call, or should we fix that API?
> >> If the device remove API needs to be used, what is the point of
> >> 'rte_rawdev_pmd_release()' API?
> >>
> >> cc'ed rawdev maintainers for comment.
> >
> > Yes, this patch is to release all the resources of ifpga_rawdev after
> > testpmd
> exit, the not released resources are interrupt and thread.
> >
> > rte_rawdev_pmd_release implemented in ifpga_rawdev only release
> memory allocated by ifpga driver, that's the expected behavior.
> >
> > I think it's a simple and safe way to release resources completely by
> > calling
> rte_dev_remove.
> >
>
> If device hot remove is better option, why 'rte_rawdev_pmd_release()' API
> exists?
Agree, let me try to free all resources in rte_rawdev_pmd_release().
>
> >>
> >>> Fixes: f724a802 ("raw/ifpga: add miscellaneous APIs")
> >>>
> >>> Signed-off-by: Wei Huang
> >>> ---
> >>>drivers/raw/ifpga/ifpga_rawdev.c | 4 +++-
> >>>1 file changed, 3 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/drivers/raw/ifpga/ifpga_rawdev.c
> >>> b/drivers/raw/ifpga/ifpga_rawdev.c
> >>> index fdf3c23..88c38aa 100644
> >>> --- a/drivers/raw/ifpga/ifpga_rawdev.c
> >>> +++ b/drivers/raw/ifpga/ifpga_rawdev.c
> >>> @@ -1787,12 +1787,14 @@ int ifpga_rawdev_partial_reconfigure(struct
> >> rte_rawdev *dev, int port,
> >>>void ifpga_rawdev_cleanup(void)
> >>>{
> >>> struct ifpga_rawdev *dev;
> >>> + struct rte_rawdev *rdev;
> >>> unsigned int i;
> >>>
> >>> for (i = 0; i < IFPGA_RAWDEV_NUM; i++) {
> >>> dev = &ifpga_rawdevices[i];
> >>> if (dev->rawdev) {
> >>> - rte_rawdev_pmd_release(dev->rawdev);
> >>> + rdev = dev->rawdev;
> >>> + rte_dev_remove(rdev->device);
> >>> dev->rawdev = NULL;
> >>> }
> >>> }
> >
Re: [dpdk-dev] [dpdk-stable] [PATCH v11 1/4] raw/ifpga: add fpga rsu function
Hi Ferruh, Cyborg is an OpenStack project that aims to provide a general purpose management framework for acceleration resources (i.e. various types of accelerators such as GPU, FPGA, NP, ODP, DPDK/SPDK and so on). To update the FPGA flash is one of requirements from Cyborg. Originally there are no such interfaces, so I added them. These interfaces use rte_rawdev to identify which FPGA to access, they will be called in opae_update_flash(), opae_cancel_flash_update() and opae_reboot_device() in ifpga_opae_api.c . These opae_xxx function use PCI address to identify FPGA instead of rte_rawdev, so the caller has no need to know the existence of rte_rawdev. In fact, Cyborg is Python application, these opae_xxx functions will be eventually wrapped in a Python module for Cyborg to call. Thanks, Wei -Original Message- From: Ferruh Yigit Sent: Friday, January 22, 2021 00:30 To: Huang, Wei ; dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z Cc: sta...@dpdk.org; Zhang, Tianfei ; Ray Kinsella Subject: Re: [dpdk-stable] [PATCH v11 1/4] raw/ifpga: add fpga rsu function On 1/21/2021 6:03 AM, Wei Huang wrote: > RSU (Remote System Update) depends on secure manager which may be > different on various implementations, so a new secure manager device > is implemented for adapting such difference. > There are three major functions added: > 1. ifpga_rawdev_update_flash() updates flash with specific image file. > 2. ifpga_rawdev_stop_flash_update() aborts flash update process. > 3. ifpga_rawdev_reload() reloads FPGA from updated flash. > > Signed-off-by: Wei Huang > Acked-by: Tianfei Zhang > Acked-by: Rosen Xu <...> > @@ -76,4 +76,9 @@ int > ifpga_unregister_msix_irq(enum ifpga_irq_type type, > int vec_start, rte_intr_callback_fn handler, void *arg); > > +int ifpga_rawdev_update_flash(struct rte_rawdev *dev, const char *image, > + uint64_t *status); > +int ifpga_rawdev_stop_flash_update(struct rte_rawdev *dev, int > +force); int ifpga_rawdev_reload(struct rte_rawdev *dev, int type, int > +page); > + > #endif /* _IFPGA_RAWDEV_H_ */ > Hi Wei, Please help me understand the rawdev, who should be calling the above newly added functions?
Re: [dpdk-dev] [PATCH v11 3/4] raw/ifpga: add OPAE API for OpenStack Cyborg
Hi Ferruh,
These log information will not be output for the default log level is
OPAE_LOG_ERR, they are output only when log level is set to OPAE_LOG_INFO. I
can remove them if you think it's necessary.
All these opae_xxx functions will check the input PCI address is valid or not,
and the rawdev PMD will also filter input PCI device ID. I think it's enough to
prevent these function to be misused.
Thanks,
Wei
-Original Message-
From: Ferruh Yigit
Sent: Friday, January 22, 2021 00:30
To: Huang, Wei ; dev@dpdk.org; Xu, Rosen
; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei
Subject: Re: [dpdk-dev] [PATCH v11 3/4] raw/ifpga: add OPAE API for OpenStack
Cyborg
On 1/21/2021 6:03 AM, Wei Huang wrote:
> Cyborg is an OpenStack project that aims to provide a general purpose
> management framework for acceleration resources (i.e. various types of
> accelerators such as GPU, FPGA, NP, ODP, DPDK/SPDK and so on).
> It needs some OPAE type APIs to manage PACs (Programmable Acceleration
> Card) with Intel FPGA. Below major functions are added to meets Cyborg
> requirements.
> 1. opae_init() set up OPAE environment.
> 2. opae_cleanup() clean up OPAE environment.
> 3. opae_enumerate() searches PAC with specific FPGA.
> 4. opae_get_property() gets properties of FPGA.
> 5. opae_partial_reconfigure() perform partial configuration on FPGA.
> 6. opae_get_image_info() gets information of image file.
> 7. opae_update_flash() updates FPGA flash with specific image file.
> 8. opae_cancel_flash_update() cancel process of FPGA flash update.
> 9. opae_probe_device() manually probe specific FPGA with ifpga driver.
> 10. opae_remove_device() manually remove specific FPGA from ifpga driver.
> 11. opae_bind_driver() binds specific FPGA with specified kernel driver.
> 12. opae_unbind_driver() unbinds specific FPGA from kernel driver.
> 13. opae_reboot_device() reboots specific FPGA (do reconfiguration).
>
> Signed-off-by: Wei Huang
> Acked-by: Tianfei Zhang
> Acked-by: Rosen Xu
<...>
> +
> +RTE_INIT(init_api_env)
> +{
> + eal_inited = 0;
> + opae_log_level = OPAE_LOG_ERR;
> + opae_log_file = NULL;
> + ifpga_rawdev_logtype = 0;
> +
> + opae_log_info("API environment is initialized\n"); }
> +
> +RTE_FINI(clean_api_env)
> +{
> + if (opae_log_file) {
> + fclose(opae_log_file);
> + opae_log_file = NULL;
> + }
> + opae_log_info("API environment is cleaned\n"); }
> +
There are called in constructor and destructor, which means all DPDK
applications have this PMD compiled will run above and print some log, is this
really what we want here?
Is there a way to limit the initialize to the PMD context, like 'probe()' etc?
Re: [dpdk-dev] [PATCH v11 3/4] raw/ifpga: add OPAE API for OpenStack Cyborg
Hi Ferruh,
That's a good question.
The answer is YES, we do need all these functions. We have complete the
integrated test with Cyborg, there is no redundant function. Let me show you a
use case in Cyborg.
Cyborg will update FPGA flash and reboot it to make it effective, so they will
call functions like below.
1. opae_enumerate() to find the target FPGA
2. opae_get_property() to check FPGA version
3. opae_get_image_info() to check the update image
4. opae_update_flash() to update FPGA flash
5. opae_reboot_device() to reboot FPGA
6. After reboot, FPGA kernel driver will not be vfio-pci by default,
opae_bind_driver() is used to bind vfio-pci kernel driver.
7. opae_probe_device() to attach ifpga PMD to FPGA, then this FPGA can be
managed by Cyborg again.
These functions will be wrapped in Python package. Cyborg require this Python
package can be downloaded from PyPI and compiled without DPDK installed.
So there is an independent project which will create a static library file from
target DPDK. This library is part of Python package and will be used when
compiling Python module. That's why I didn't export these function in map file.
BTW, the header file ifpga_opae_api.h is also integrated into Python package
from target DPDK.
Thanks,
Wei
-Original Message-
From: Ferruh Yigit
Sent: Friday, January 22, 2021 00:30
To: Huang, Wei ; dev@dpdk.org; Xu, Rosen
; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei ; Ray Kinsella
Subject: Re: [dpdk-dev] [PATCH v11 3/4] raw/ifpga: add OPAE API for OpenStack
Cyborg
On 1/21/2021 6:03 AM, Wei Huang wrote:
> Cyborg is an OpenStack project that aims to provide a general purpose
> management framework for acceleration resources (i.e. various types of
> accelerators such as GPU, FPGA, NP, ODP, DPDK/SPDK and so on).
> It needs some OPAE type APIs to manage PACs (Programmable Acceleration
> Card) with Intel FPGA. Below major functions are added to meets Cyborg
> requirements.
> 1. opae_init() set up OPAE environment.
> 2. opae_cleanup() clean up OPAE environment.
> 3. opae_enumerate() searches PAC with specific FPGA.
> 4. opae_get_property() gets properties of FPGA.
> 5. opae_partial_reconfigure() perform partial configuration on FPGA.
> 6. opae_get_image_info() gets information of image file.
> 7. opae_update_flash() updates FPGA flash with specific image file.
> 8. opae_cancel_flash_update() cancel process of FPGA flash update.
> 9. opae_probe_device() manually probe specific FPGA with ifpga driver.
> 10. opae_remove_device() manually remove specific FPGA from ifpga driver.
> 11. opae_bind_driver() binds specific FPGA with specified kernel driver.
> 12. opae_unbind_driver() unbinds specific FPGA from kernel driver.
> 13. opae_reboot_device() reboots specific FPGA (do reconfiguration).
>
Hi Wei,
As far as I understand you are adding above public functions which are on top
of raw/ifpga driver functions, so they are like PMD specific APIs, I think
there are a few problems with it:
1) Do we really need/want this much PMD specific API? Can't we have them
through the rawdev abstraction layer?
2) DPDK public APIs are part of API/ABI policy, so there are a few rules they
have to follow, like:
- They should start with 'rte_' prefix, and the PMD specific APIs should start
with 'rte_pmd_' prefix
- They should be in the .map file
- They should be experimental at least one release
- They should be fully documented in a doxygen format
- Header file should be added to index file for API documentation
Please don't update above before 1) is clearified and we are sure new APIs are
required.
<...>
> @@ -13,8 +13,10 @@ objs = [base_objs]
> deps += ['ethdev', 'rawdev', 'pci', 'bus_pci', 'kvargs',
> 'bus_vdev', 'bus_ifpga', 'net', 'net_i40e', 'net_ipn3ke']
>
> -sources = files('ifpga_rawdev.c')
> +sources = files('ifpga_rawdev.c', 'ifpga_opae_api.c')
>
> includes += include_directories('base')
> includes += include_directories('../../net/ipn3ke')
> includes += include_directories('../../net/i40e')
> +
> +install_headers('ifpga_opae_api.h')
>
There is a 'headers' helper that you can use for meson. Also the header file
name should start with 'rte_pmd_'.
Even before this patch, isn't application has to include the rawdev PMD header?
Why that header was not installed?
Re: [dpdk-dev] [PATCH v11 3/4] raw/ifpga: add OPAE API for OpenStack Cyborg
Hi Ferruh,
Most of other APIs depends on EAL environment, they will make sure EAL is ready
be executing their task.
As replied in previous mail, these functions will be called in Python
environment. Python application should initialize EAL explicitly before calling
other APIs like opae_update_flash. If application forget to do such
initialization, an error information will be output when calling other APIs.
Thanks,
Wei
-Original Message-
From: Ferruh Yigit
Sent: Friday, January 22, 2021 00:34
To: Huang, Wei ; dev@dpdk.org; Xu, Rosen
; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei
Subject: Re: [dpdk-dev] [PATCH v11 3/4] raw/ifpga: add OPAE API for OpenStack
Cyborg
On 1/21/2021 6:03 AM, Wei Huang wrote:
> +int opae_init(int eal_init_result)
> +{
> + int ret = 0;
> +
> + if (!check_eal(0))
> + return 0;
> +
> + if (eal_init_result < 0) {
> + if (rte_errno == EALREADY) {
> + eal_inited = 1;
> + opae_log_info("EAL already initialized\n");
> + } else {
> + opae_log_err("Cannot initialize EAL\n");
> + ret = -1;
> + }
> + } else {
> + eal_inited = 1;
> + opae_log_info("Initialize EAL done\n");
> + }
> +
> + return ret;
> +}
Why an PMD API needs to know eal initialization status? This may be pointing a
bad design, is it possible that most of the 'opae_' API code should go to the
application?
Re: [dpdk-dev] [dpdk-stable] [PATCH v11 4/4] examples/ifpga: add example for opae ifpga API
Hi Ferruh, Yes, all the command need PCI BDF as identifier of FPGA, because Cyborg use PCI BDF to call these APIs, Cyborg does not see any DPDK data structures. That's why I add a group of middle layer functions in ifpga_opae_api.c to isolate Cyborg from DPDK. Thanks, Wei -Original Message- From: Ferruh Yigit Sent: Friday, January 22, 2021 00:49 To: Huang, Wei ; dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z Cc: sta...@dpdk.org; Zhang, Tianfei Subject: Re: [dpdk-stable] [PATCH v11 4/4] examples/ifpga: add example for opae ifpga API On 1/21/2021 6:03 AM, Wei Huang wrote: > +get_property command > + > + > +Display property information of specified FPGA. Property type is defined as > below. > +0 - All properties > +1 - PCI property > +2 - FME property > +4 - port property > +8 - BMC property > +PCI property is always available, other properties can only be > +displayed after ifpga driver is probed to the FPGA. > + > +.. code-block:: console > + > + opae> get_property 24:00.0 0 > + PCI: > +PCIe s:b:d.f : :24:00.0 > +kernel driver: vfio-pci > + FME: > +platform : Vista Creek > +DCP version : DCP 1.2 > +phase: Beta > +interface: 2x2x25G > +build version: 0.0.2 > +ports num: 1 > +boot page: user > +pr interface id : a5d72a3c-c8b0-4939-912c-f715e5dc10ca > + PORT0: > +access type : PF > +accelerator id : 8892c23e-2eed-4b44-8bb6-5c88606e07df > + BMC: > +MAX10 version: D.2.0.5 > +NIOS FW version : D.2.0.12 Are all commands interacting device need to provide the PCI BDF as parameter to select the device? Like '24:00.0' in above sample. Why not get the unique rawdev identifier, like port_id equivalent of the ethdev, and use it in the command line, won't it be easier?
Re: [dpdk-dev] [dpdk-stable] [PATCH v11 1/4] raw/ifpga: add fpga rsu function
Hi, -Original Message- From: Xu, Rosen Sent: Monday, January 25, 2021 18:33 To: Yigit, Ferruh ; Huang, Wei ; dev@dpdk.org; Zhang, Qi Z Cc: sta...@dpdk.org; Zhang, Tianfei ; Ray Kinsella Subject: RE: [dpdk-stable] [PATCH v11 1/4] raw/ifpga: add fpga rsu function Hi, > -Original Message- > From: Ferruh Yigit > Sent: Monday, January 25, 2021 18:01 > To: Huang, Wei ; dev@dpdk.org; Xu, Rosen > ; Zhang, Qi Z > Cc: sta...@dpdk.org; Zhang, Tianfei ; Ray > Kinsella > Subject: Re: [dpdk-stable] [PATCH v11 1/4] raw/ifpga: add fpga rsu > function > > On 1/22/2021 2:18 AM, Huang, Wei wrote: > > > > >> > >> -Original Message- > >> From: Ferruh Yigit > >> Sent: Friday, January 22, 2021 00:30 > >> To: Huang, Wei ; dev@dpdk.org; Xu, Rosen > >> ; Zhang, Qi Z > >> Cc: sta...@dpdk.org; Zhang, Tianfei ; Ray > >> Kinsella > >> Subject: Re: [dpdk-stable] [PATCH v11 1/4] raw/ifpga: add fpga rsu > >> function > >> > >> On 1/21/2021 6:03 AM, Wei Huang wrote: > >>> RSU (Remote System Update) depends on secure manager which may be > >>> different on various implementations, so a new secure manager > >>> device is implemented for adapting such difference. > >>> There are three major functions added: > >>> 1. ifpga_rawdev_update_flash() updates flash with specific image file. > >>> 2. ifpga_rawdev_stop_flash_update() aborts flash update process. > >>> 3. ifpga_rawdev_reload() reloads FPGA from updated flash. > >>> > >>> Signed-off-by: Wei Huang > >>> Acked-by: Tianfei Zhang > >>> Acked-by: Rosen Xu > >> > >> <...> > >> > >>> @@ -76,4 +76,9 @@ int > >>>ifpga_unregister_msix_irq(enum ifpga_irq_type type, > >>>int vec_start, rte_intr_callback_fn handler, void *arg); > >>> > >>> +int ifpga_rawdev_update_flash(struct rte_rawdev *dev, const char > >>> +*image, uint64_t *status); int > >>> +ifpga_rawdev_stop_flash_update(struct rte_rawdev *dev, int > >>> +force); int ifpga_rawdev_reload(struct rte_rawdev *dev, int type, > >>> +int page); > >>> + > >>>#endif /* _IFPGA_RAWDEV_H_ */ > >>> > >> > >> Hi Wei, > >> > >> Please help me understand the rawdev, who should be calling the > >> above > newly added functions? > >> > > > > Hi Ferruh, > > > > Cyborg is an OpenStack project that aims to provide a general > purpose management framework for acceleration resources (i.e. various > types of accelerators such as GPU, FPGA, NP, ODP, DPDK/SPDK and so on). > > > > To update the FPGA flash is one of requirements from Cyborg. > Originally there are no such interfaces, so I added them. > > > > These interfaces use rte_rawdev to identify which FPGA to access, > they will be called in opae_update_flash(), > > opae_cancel_flash_update() and > opae_reboot_device() in ifpga_opae_api.c . > > > > These opae_xxx function use PCI address to identify FPGA instead of > rte_rawdev, so the caller has no need to know the existence of rte_rawdev. > > In fact, Cyborg is Python application, these opae_xxx functions > will be eventually wrapped in a Python module for Cyborg to call. > > > > Thanks for clarification, I see what you are doing, but still I think > these APIs are not belong to a driver, they look like more application level. > > @Wei, @Rosen, what do you think to keep only generic raw/ipfga APIs in > the driver and move all 'ifpga_opae_api.c/h' to the sample application? @Huang, Wei, could you classify which functions are only used in sample application? All the opae_xxx functions used in sample application will be used by Cyborg. I will try to move ifpga_opae_api.c/h to sample according to Ferruh's comments. > In that case raw/ifpga APIs still can get the port_id as parameter, > and the 'opae' layer in the sample can do the conversion?
Re: [dpdk-dev] [PATCH v12 1/4] raw/ifpga: add fpga rsu function
-Original Message-
From: Ferruh Yigit
Sent: Thursday, January 28, 2021 21:25
To: Huang, Wei ; dev@dpdk.org; Xu, Rosen
; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei ; Ray Kinsella
; Hemant Agrawal
Subject: Re: [dpdk-dev] [PATCH v12 1/4] raw/ifpga: add fpga rsu function
On 1/26/2021 6:45 AM, Wei Huang wrote:
> RSU (Remote System Update) depends on secure manager which may be
> different on various implementations, so a new secure manager device
> is implemented for adapting such difference.
> There are three major functions added:
> 1. ifpga_rawdev_update_flash() updates flash with specific image file.
> 2. ifpga_rawdev_stop_flash_update() aborts flash update process.
> 3. ifpga_rawdev_reload() reloads FPGA from updated flash.
>
> Signed-off-by: Wei Huang
> Acked-by: Tianfei Zhang
> Acked-by: Rosen Xu
> ---
> v2: fix coding style issue in ifpga_fme_rsu.c and ifpga_sec_mgr.c
> ---
> v3: fix compilation issues in ifpga_fme_rsu.c
> ---
> v4: fix compilation issues in opae_intel_max10.c
> ---
<...>
> @@ -43,7 +43,7 @@ enum ifpga_rawdev_device_state {
> static inline struct opae_adapter *
> ifpga_rawdev_get_priv(const struct rte_rawdev *rawdev)
> {
> - return rawdev->dev_private;
> + return (struct opae_adapter *)rawdev->dev_private;
> }
>
> #define IFPGA_RAWDEV_MSIX_IRQ_NUM 7
> @@ -76,4 +76,9 @@ int
> ifpga_unregister_msix_irq(enum ifpga_irq_type type,
> int vec_start, rte_intr_callback_fn handler, void *arg);
>
> +int ifpga_rawdev_update_flash(struct rte_rawdev *dev, const char *image,
> + uint64_t *status);
> +int ifpga_rawdev_stop_flash_update(struct rte_rawdev *dev, int
> +force); int ifpga_rawdev_reload(struct rte_rawdev *dev, int type, int
> +page);
> +
> #endif /* _IFPGA_RAWDEV_H_ */
>
Hi Wei, Rosen,
This patch introduces three new PMD specific APIs, adding new API has some
requirements.
1)
There must be a header file for user application to include, that has the
definitions of the APIs.
This header file should be installed in meson via "headers = ..." syntax.
Indeed for rawdev a header always should be installed, because of the rawdev
design, the user application should know about the some driver structures, to
share those structures PMD should provide a header. This header seems missing.
You can start for providing the missing header, even before this patch.
Header file should be named as 'rte_pmd_.', and since it is a public header
now it should be fully documented via doxygen comments.
This header file should be added to 'doc/api/doxy-api-index.md' for API
documentathion.
OK, I will add rte_pmd_ifpga.h.
2)
All APIs should start with 'rte_pmd_' prefix.
OK, I will use 'rte_pmd_' prefix before driver APIs.
3)
All APIs should be in the .map file
OK, I will add them into .map file.
4)
Since these are new APIs, they should be marked as experimental. This is done
both documenting this in the doxygen comments and marking the function
decleration via '__rte_experimental' tag
How to document it in doxygen comments ? Could you give an example ?
5)
Please don't use "struct rte_rawdev *dev" as a argument in the APIs, since that
structure is rawdev internal structures, applications (that will call your API)
should not know or access to this struct.
Instead you can you 'dev_id' (ethdev 'port_id' equivalent) in the API, as done
in the rawdev APIs. Driver can easily get the rawdevice from 'dev_id'.
OK, I will use dev_id in driver APIs' argument.
cc'ed Ray & Hemant in case I missed something related to rawdev and API/ABIs.
And for the ifpga implementation, it is hard for me to review it, I trust Rosen
for it.
Re: [dpdk-dev] [dpdk-stable] [PATCH v12 3/4] raw/ifpga: add fpga helper function
Hi, -Original Message- From: Ferruh Yigit Sent: Thursday, January 28, 2021 21:30 To: Huang, Wei ; dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z Cc: sta...@dpdk.org; Zhang, Tianfei Subject: Re: [dpdk-stable] [PATCH v12 3/4] raw/ifpga: add fpga helper function On 1/26/2021 6:45 AM, Wei Huang wrote: > Below helper functions are used to implement OPAE application. > 1. ifpga_rawdev_lock() and ifpga_rawdev_unlock() protect FPGA access > in multiproces. Why driver needs to expose the lock APIs to the application? Can't application use its own locking mechanism for multiprocess, and driver can use its own locking for its needs. Driver exposing locking internals to applications doesn't look right, but I don't know the full usecase. I agree, these two lock APIs will be used only in rte_pmd_ifpga.c in next patch, application cannot see them. > 2. ifpga_rawdev_get_rsu_stat() and ifpga_rawdev_set_rsu_stat() record > current RSU status and progress. > 3. ifpga_rawdev_get_phy_info() get information of connected retimer. > 4. ifpga_rawdev_partial_reconfigure() do partial reconfiguration. > 5. ifpga_rawdev_cleanup() free software resources allocated for driver. > > Signed-off-by: Wei Huang > Acked-by: Tianfei Zhang > Acked-by: Rosen Xu <...>
Re: [dpdk-dev] [PATCH v12 4/4] examples/ifpga: add example for ifpga opae API
Hi, -Original Message- From: Ferruh Yigit Sent: Thursday, January 28, 2021 21:35 To: Huang, Wei ; dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z Cc: sta...@dpdk.org; Zhang, Tianfei Subject: Re: [dpdk-dev] [PATCH v12 4/4] examples/ifpga: add example for ifpga opae API On 1/26/2021 6:45 AM, Wei Huang wrote: > Below major OPAE APIs are added in this example. > 1. opae_init_eal() set up EAL environment. > 2. opae_cleanup_eal() clean up EAL environment. > 3. opae_enumerate() searches PAC with specific FPGA. > 4. opae_get_property() gets properties of FPGA. > 5. opae_partial_reconfigure() perform partial configuration on FPGA. > 6. opae_get_image_info() gets information of image file. > 7. opae_update_flash() updates FPGA flash with specific image file. > 8. opae_cancel_flash_update() cancel process of FPGA flash update. > 9. opae_probe_device() manually probe specific FPGA with ifpga driver. > 10. opae_remove_device() manually remove specific FPGA from ifpga driver. > 11. opae_bind_driver() binds specific FPGA with specified kernel driver. > 12. opae_unbind_driver() unbinds specific FPGA from kernel driver. > 13. opae_reboot_device() reboots specific FPGA. > > An example application shows how to call above OPAE APIs. > You can test each API by running corresponding command. > A guide is also added to show how to run the example. > > Signed-off-by: Wei Huang > Acked-by: Tianfei Zhang > Acked-by: Rosen Xu > --- > v2: fix coding style issue in commands.c > --- > v3: add guide for running example > --- > v4: fix compilation issue of ifpga.rst > --- > v5: add ifpga.rst into sample_app_ug/index.rst > --- > v6: implement OPAE APIs in example instead of ifpga rawdev > --- > MAINTAINERS|1 + > doc/guides/sample_app_ug/ifpga.rst | 433 +++ > doc/guides/sample_app_ug/index.rst |1 + > examples/ifpga/Makefile| 45 + > examples/ifpga/commands.c | 1321 > examples/ifpga/commands.h | 16 + > examples/ifpga/main.c | 38 + > examples/ifpga/meson.build | 20 + > examples/ifpga/opae_api.c | 1788 > examples/ifpga/opae_api.h | 245 > 10 files changed, 3908 insertions(+) > create mode 100644 doc/guides/sample_app_ug/ifpga.rst > create mode 100644 examples/ifpga/Makefile > create mode 100644 examples/ifpga/commands.c > create mode 100644 examples/ifpga/commands.h > create mode 100644 examples/ifpga/main.c > create mode 100644 examples/ifpga/meson.build > create mode 100644 examples/ifpga/opae_api.c > create mode 100644 examples/ifpga/opae_api.h > > diff --git a/MAINTAINERS b/MAINTAINERS index eafe9f8c4..04a8fdebb > 100644 > --- a/MAINTAINERS > +++ b/MAINTAINERS > @@ -1237,6 +1237,7 @@ M: Tianfei zhang > T: git://dpdk.org/next/dpdk-next-net-intel > F: drivers/raw/ifpga/ > F: doc/guides/rawdevs/ifpga.rst > +F: doc/guides/sample_app_ug/ifpga.rst The new example also needs to be added to the maintainers file. OK, I will add it maintainers file. > > IOAT Rawdev > M: Bruce Richardson diff --git > a/doc/guides/sample_app_ug/ifpga.rst > b/doc/guides/sample_app_ug/ifpga.rst > new file mode 100644 > index 0..adcac3bdf > --- /dev/null > +++ b/doc/guides/sample_app_ug/ifpga.rst > @@ -0,0 +1,433 @@ > +.. SPDX-License-Identifier: BSD-3-Clause > +Copyright(c) 2020-2021 Intel Corporation. > + > +Intel FPGA Sample Application > += > + > +The Intel FPGA sample application is an example of how to use OPAE > +API to manage Intel FPGA. > + Can you please explain more what is "OPAE API" in documentation, provide links if possible. OK, I will add "OPAE" description and link in documentation. > +Overview > + > + > +The Intel FPGA sample application is a simple application that > +demonstrates the use of the OPAE API provided by ifpga driver in the DPDK. > +This application is a readline-like interface that can be used to > +manage Intel FPGA, in a Linux* application environment. > + > +Compiling the Application > +- > + > +To compile the sample application see :doc:`compiling` > + > +The application is located in the ``ifpga`` sub-directory. > + > +Running the Application > +--- > + > +To run the application in linux environment, issue the following command: > + > +.. code-block:: console > + > +$ .//examples/dpdk-ifpga --proc-type=auto > + > +Refer to the *DPDK Getting Started Guide* for general information on > +running applications and the Environment Abstraction Layer (EAL) o
Re: [dpdk-dev] [dpdk-stable] [PATCH v13 0/4] raw/ifpga: add extra OPAE APIs
-Original Message- From: Thomas Monjalon Sent: Wednesday, February 10, 2021 17:38 To: Huang, Wei Cc: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z ; sta...@dpdk.org; Zhang, Tianfei ; Yigit, Ferruh Subject: Re: [dpdk-stable] [PATCH v13 0/4] raw/ifpga: add extra OPAE APIs 10/02/2021 02:48, Wei Huang: > Cyborg is part of OpenStack, it needs some OPAE APIs to manage devices > with Intel FPGA. The first three patches implement extra APIs to meet > Cyborg requirement. The last patch add an example to show how to use > these APIs. > > Main changes from v12: > - implement ifpga APIs in rte_pmd_ifpga.c > > Wei Huang (4): > raw/ifpga: add fpga rsu APIs > raw/ifpga: add APIs to get fpga information > raw/ifpga: add miscellaneous APIs > examples/ifpga: add example for ifpga APIs Not sure we want to add an example for a driver-specific API. Thomas, this example has two purposes, one is for how to use ifpga API, the other is to create static library for Cyborg application. PS: please use --in-reply-to to keep all versions in the same thread.
RE: [PATCH v2 12/20] raw/ifpga: inherit lock annotations
Hi,
> -Original Message-
> From: Xu, Rosen
> Sent: Monday, February 27, 2023 14:29
> To: David Marchand ; dev@dpdk.org
> Cc: tho...@monjalon.net; Zhang, Tianfei ; Huang,
> Wei
> Subject: RE: [PATCH v2 12/20] raw/ifpga: inherit lock annotations
>
> Hi,
>
> > -Original Message-
> > From: David Marchand
> > Sent: Friday, February 24, 2023 11:12 PM
> > To: dev@dpdk.org
> > Cc: tho...@monjalon.net; Xu, Rosen ; Zhang,
> > Tianfei
> > Subject: [PATCH v2 12/20] raw/ifpga: inherit lock annotations
> >
> > The checks in those helpers are useless:
> > - all (start/stop/reset/test) callers ensure that dev != NULL,
> > - dev->sd can't be NULL either as it would mean the application is calling
> > those helpers for a dev pointer that did not pass initialisation,
> >
> > Once the checks are removed, the only thing that remains is calls to
> > the rte_spinlock API, so simply use macros and inherit annotations
> > from the lock API.
> >
> > Signed-off-by: David Marchand
> > ---
> > drivers/raw/ifpga/afu_pmd_core.c | 17 ++---
> > 1 file changed, 2 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/raw/ifpga/afu_pmd_core.c
> > b/drivers/raw/ifpga/afu_pmd_core.c
> > index ddf7a34f33..3ab1f47ac1 100644
> > --- a/drivers/raw/ifpga/afu_pmd_core.c
> > +++ b/drivers/raw/ifpga/afu_pmd_core.c
> > @@ -23,21 +23,8 @@ static struct rte_afu_uuid
> > afu_pmd_uuid_map[AFU_RAWDEV_MAX_DRVS+1];
> > TAILQ_HEAD(afu_drv_list, afu_rawdev_drv); static struct afu_drv_list
> > afu_pmd_list = TAILQ_HEAD_INITIALIZER(afu_pmd_list);
> >
> > -static inline int afu_rawdev_trylock(struct afu_rawdev *dev) -{
> > - if (!dev || !dev->sd)
> > - return 0;
> > -
> > - return rte_spinlock_trylock(&dev->sd->lock);
> > -}
> > -
> > -static inline void afu_rawdev_unlock(struct afu_rawdev *dev) -{
> > - if (!dev || !dev->sd)
> > - return;
> > -
> > - rte_spinlock_unlock(&dev->sd->lock);
> > -}
> > +#define afu_rawdev_trylock(dev) rte_spinlock_trylock(&dev->sd->lock)
> > +#define afu_rawdev_unlock(dev) rte_spinlock_unlock(&dev->sd->lock)
> >
> > static int afu_rawdev_configure(const struct rte_rawdev *rawdev,
> > rte_rawdev_obj_t config, size_t config_size)
> > --
> > 2.39.2
>
>
> It looks good for me.
> Reviewed-by: Rosen Xu
Agree that pointer check is redundant, rte_spinlock API can be called directly.
Reviewed-by: Wei Huang
RE: [PATCH v1] raw/ifpga: remove virtual device unplug operation
Hi Tomas, > -Original Message- > From: Thomas Monjalon > Sent: Tuesday, March 21, 2023 00:06 > To: Huang, Wei > Cc: dev@dpdk.org; david.march...@redhat.com; sta...@dpdk.org; Xu, > Rosen ; Zhang, Tianfei ; > Zhang, Qi Z > Subject: Re: [PATCH v1] raw/ifpga: remove virtual device unplug operation > > 16/03/2023 21:44, Wei Huang: > > VDEV bus has implemented cleanup() function to perform cleanup for > > devices on the bus during eal_cleanup(), so there is no need for ifpga > > driver to record virtual devices and unplug them. > > Why no need? > If the application wants to explicitly remove a device, what happens? > > EAL will output an error information "Cannot find plugged device (%s)".
RE: [PATCH v1] raw/ifpga: remove virtual device unplug operation
Hi Thomas, > -Original Message- > From: Thomas Monjalon > Sent: Tuesday, March 21, 2023 16:14 > To: Huang, Wei > Cc: dev@dpdk.org; david.march...@redhat.com; sta...@dpdk.org; Xu, > Rosen ; Zhang, Tianfei ; > Zhang, Qi Z > Subject: Re: [PATCH v1] raw/ifpga: remove virtual device unplug operation > > 21/03/2023 01:11, Huang, Wei: > > From: Thomas Monjalon > > > 16/03/2023 21:44, Wei Huang: > > > > VDEV bus has implemented cleanup() function to perform cleanup for > > > > devices on the bus during eal_cleanup(), so there is no need for > > > > ifpga driver to record virtual devices and unplug them. > > > > > > Why no need? > > > If the application wants to explicitly remove a device, what happens? > > > > > > > > EAL will output an error information "Cannot find plugged device (%s)". > > It does not look what we expect. > Let me clear it. With this patch, no error information will be outputted. Without this patch, error information will be outputted. Because bus cleanup action will unplug virtual device, then ifpga PMD unplug the virtual device which is already be cleanup, bus->find_device() returns NULL, EAL output "Cannot find plugged device (%s)\n" at line 302 in eal_common_dev.c
RE: [PATCH v1] raw/ifpga: remove virtual device unplug operation
> -Original Message- > From: Thomas Monjalon > Sent: Tuesday, March 21, 2023 18:30 > To: Huang, Wei > Cc: dev@dpdk.org; david.march...@redhat.com; sta...@dpdk.org; Xu, > Rosen ; Zhang, Tianfei ; > Zhang, Qi Z > Subject: Re: [PATCH v1] raw/ifpga: remove virtual device unplug operation > > 21/03/2023 09:41, Huang, Wei: > > From: Thomas Monjalon > > > 21/03/2023 01:11, Huang, Wei: > > > > From: Thomas Monjalon > > > > > 16/03/2023 21:44, Wei Huang: > > > > > > VDEV bus has implemented cleanup() function to perform cleanup > > > > > > for devices on the bus during eal_cleanup(), so there is no > > > > > > need for ifpga driver to record virtual devices and unplug them. > > > > > > > > > > Why no need? > > > > > If the application wants to explicitly remove a device, what happens? > > > > > > > > > > > > > > EAL will output an error information "Cannot find plugged device (%s)". > > > > > > It does not look what we expect. > > > > > Let me clear it. > > With this patch, no error information will be outputted. > > Without this patch, error information will be outputted. > > Because bus cleanup action will unplug virtual device, then ifpga PMD > > unplug the virtual device which is already be cleanup, > > Why ipfga unplug the device after the bus cleanup? > I'm not following. > The virtual device is created upon ifpga, if VDEV bus doesn't perform cleanup, ifpga has the responsibility to unplug these virtual devices. > > bus->find_device() returns NULL, > > EAL output "Cannot find plugged device (%s)\n" at line 302 in > > eal_common_dev.c > > Anyway, the good answer is not to completely remove the "remove" > operation. > If not to completely remove the "remove", the same virtual device will be unplug twice, is it reasonable?
RE: [PATCH v1] raw/ifpga: remove virtual device unplug operation
> -Original Message-
> From: Thomas Monjalon
> Sent: Wednesday, March 22, 2023 19:54
> To: Huang, Wei
> Cc: dev@dpdk.org; david.march...@redhat.com; sta...@dpdk.org; Xu,
> Rosen ; Zhang, Tianfei ;
> Zhang, Qi Z
> Subject: Re: [PATCH v1] raw/ifpga: remove virtual device unplug operation
>
> 22/03/2023 02:26, Huang, Wei:
> > From: Thomas Monjalon
> > > 21/03/2023 09:41, Huang, Wei:
> > > > From: Thomas Monjalon
> > > > > 21/03/2023 01:11, Huang, Wei:
> > > > > > From: Thomas Monjalon
> > > > > > > 16/03/2023 21:44, Wei Huang:
> > > > > > > > VDEV bus has implemented cleanup() function to perform
> > > > > > > > cleanup for devices on the bus during eal_cleanup(), so
> > > > > > > > there is no need for ifpga driver to record virtual devices and
> unplug them.
> > > > > > >
> > > > > > > Why no need?
> > > > > > > If the application wants to explicitly remove a device, what
> happens?
> > > > > > >
> > > > > > >
> > > > > > EAL will output an error information "Cannot find plugged device
> (%s)".
> > > > >
> > > > > It does not look what we expect.
> > > > >
> > > > Let me clear it.
> > > > With this patch, no error information will be outputted.
> > > > Without this patch, error information will be outputted.
> > > > Because bus cleanup action will unplug virtual device, then ifpga
> > > > PMD unplug the virtual device which is already be cleanup,
> > >
> > > Why ipfga unplug the device after the bus cleanup?
> > > I'm not following.
> > >
> > The virtual device is created upon ifpga, if VDEV bus doesn't perform
> > cleanup, ifpga has the responsibility to unplug these virtual devices.
>
> Really I don't understand the flow.
> Are you talking about EAL cleanup case?
Yes, it's about EAL cleanup.
> What happens first? Do you need ifpga to be called first?
The cleanup flow is rte_eal_cleanup() -> eal_bus_cleanup()
eal_bus_cleanup() will call each bus's cleanup method to complete cleanup work.
There are three types of device related to ifpga PMD: PCI device, VDEV device
and AFU device.
VDEV device is created on PCI device, it's a mediate device which purpose is to
plug a AFU device on IFPGA bus.
Before eal_bus_cleanup() is implemented, application will call
rte_pmd_ifpga_cleanup() to remove PCI device, VDEV device will be removed when
PCI device is removed, AFU device will be removed when VDEV device is removed,
it works fine.
Now eal_bus_cleanup() takes the job, application has no need to call
rte_pmd_ifpga_cleanup(), ifpga PCI device has no need to remove ifpga VDEV
device and ifpga VDEV device has no need to remove ifpga AFU device.
> I think you need the correct checks to allow any order of cleanup.
When this patch is committed, no order dependent on cleanup.
> > > > bus->find_device() returns NULL,
> > > > EAL output "Cannot find plugged device (%s)\n" at line 302 in
> > > > eal_common_dev.c
> > >
> > > Anyway, the good answer is not to completely remove the "remove"
> > > operation.
> > >
> > If not to completely remove the "remove", the same virtual device will be
> unplug twice, is it reasonable?
>
> You need to add a check to not unplug something already unplugged.
> But you must allow the user calling "remove" directly.
>
rte_pmd_ifpga_cleanup() is the only one interface for user to calling "remove"
directly, when this patch is committed, VDEV and AFU device will not be
unplugged twice.
For PCI device, the implementation of rte_pmd_ifpga_cleanup() is like below
for (i = 0; i < IFPGA_RAWDEV_NUM; i++) {
dev = &ifpga_rawdevices[i];
if (dev->rawdev) {
rte_rawdev_pmd_release(dev->rawdev);
dev->rawdev = NULL;
}
}
If ifpga PCI device is already removed, dev->rawdev is NULL, it will not be
unplugged again.
RE: [PATCH v1] raw/ifpga: remove virtual device unplug operation
> -Original Message-
> From: Thomas Monjalon
> Sent: Thursday, March 23, 2023 16:52
> To: Huang, Wei
> Cc: dev@dpdk.org; david.march...@redhat.com; sta...@dpdk.org; Xu,
> Rosen ; Zhang, Tianfei ;
> Zhang, Qi Z
> Subject: Re: [PATCH v1] raw/ifpga: remove virtual device unplug operation
>
> 23/03/2023 04:26, Huang, Wei:
> > From: Thomas Monjalon
> > > 22/03/2023 02:26, Huang, Wei:
> > > > From: Thomas Monjalon
> > > > > 21/03/2023 09:41, Huang, Wei:
> > > > > > From: Thomas Monjalon
> > > > > > > 21/03/2023 01:11, Huang, Wei:
> > > > > > > > From: Thomas Monjalon
> > > > > > > > > 16/03/2023 21:44, Wei Huang:
> > > > > > > > > > VDEV bus has implemented cleanup() function to perform
> > > > > > > > > > cleanup for devices on the bus during eal_cleanup(),
> > > > > > > > > > so there is no need for ifpga driver to record virtual
> > > > > > > > > > devices and
> > > unplug them.
> > > > > > > > >
> > > > > > > > > Why no need?
> > > > > > > > > If the application wants to explicitly remove a device,
> > > > > > > > > what
> > > happens?
> > > > > > > > >
> > > > > > > > >
> > > > > > > > EAL will output an error information "Cannot find plugged
> > > > > > > > device
> > > (%s)".
> > > > > > >
> > > > > > > It does not look what we expect.
> > > > > > >
> > > > > > Let me clear it.
> > > > > > With this patch, no error information will be outputted.
> > > > > > Without this patch, error information will be outputted.
> > > > > > Because bus cleanup action will unplug virtual device, then
> > > > > > ifpga PMD unplug the virtual device which is already be
> > > > > > cleanup,
> > > > >
> > > > > Why ipfga unplug the device after the bus cleanup?
> > > > > I'm not following.
> > > > >
> > > > The virtual device is created upon ifpga, if VDEV bus doesn't
> > > > perform cleanup, ifpga has the responsibility to unplug these virtual
> devices.
> > >
> > > Really I don't understand the flow.
> > > Are you talking about EAL cleanup case?
> > Yes, it's about EAL cleanup.
> > > What happens first? Do you need ifpga to be called first?
> > The cleanup flow is rte_eal_cleanup() -> eal_bus_cleanup()
> > eal_bus_cleanup() will call each bus's cleanup method to complete cleanup
> work.
> > There are three types of device related to ifpga PMD: PCI device, VDEV
> device and AFU device.
> > VDEV device is created on PCI device, it's a mediate device which purpose
> is to plug a AFU device on IFPGA bus.
> > Before eal_bus_cleanup() is implemented, application will call
> rte_pmd_ifpga_cleanup() to remove PCI device, VDEV device will be
> removed when PCI device is removed, AFU device will be removed when
> VDEV device is removed, it works fine.
> > Now eal_bus_cleanup() takes the job, application has no need to call
> rte_pmd_ifpga_cleanup(), ifpga PCI device has no need to remove ifpga
> VDEV device and ifpga VDEV device has no need to remove ifpga AFU device.
>
> That's the problem in your approach.
> You need to keep the code removing children devices.
> And if all children are removed, then the parent should remove itself.
> If you implement these 2 conditions, the cleanup can happen in any order.
>
OK, this patch should be rejected, I will commit another patch according to
your comments.
> > > I think you need the correct checks to allow any order of cleanup.
> > When this patch is committed, no order dependent on cleanup.
> >
> > > > > > bus->find_device() returns NULL,
> > > > > > EAL output "Cannot find plugged device (%s)\n" at line 302 in
> > > > > > eal_common_dev.c
> > > > >
> > > > > Anyway, the good answer is not to completely remove the "remove"
> > > > > operation.
> > > > >
> > > > If not to completely remove the "remove", the same virtual device
> > > > will be
> > > unplug twice, is it reasonable?
> > >
> > > You need to add a check to not unplug something already unplugged.
> > > But you must allow the user calling "remove" directly.
> > >
> > rte_pmd_ifpga_cleanup() is the only one interface for user to calling
> > "remove" directly,
>
> No, there are functions in EAL to "remove" devices, like rte_dev_remove(),
> and we must make sure it works effectively.
>
> > when this patch is committed, VDEV and AFU device will not be unplugged
> twice.
> > For PCI device, the implementation of rte_pmd_ifpga_cleanup() is like
> below
> > for (i = 0; i < IFPGA_RAWDEV_NUM; i++) {
> > dev = &ifpga_rawdevices[i];
> > if (dev->rawdev) {
> > rte_rawdev_pmd_release(dev->rawdev);
> > dev->rawdev = NULL;
> > }
> > }
> > If ifpga PCI device is already removed, dev->rawdev is NULL, it will not be
> unplugged again.
>
>
>
RE: [PATCH v1] doc/guides/nics: remove limitation of ipn3ke
> -Original Message- > From: Thomas Monjalon > Sent: Thursday, March 23, 2023 16:53 > To: dev@dpdk.org; david.march...@redhat.com; Huang, Wei > > Cc: sta...@dpdk.org; Xu, Rosen ; Zhang, Tianfei > ; Zhang, Qi Z ; Huang, Wei > > Subject: Re: [PATCH v1] doc/guides/nics: remove limitation of ipn3ke > > 23/03/2023 02:00, Wei Huang: > > Now ipn3ke is not evaluation in 19.05 . > > What do you mean? It is not considered for evaluation since 19.05? > Do you want to backport? > No backport, this limitation is out of date, just to remove it. > [...] > > -19.05 limitation > > - > > - > > -Ipn3ke code released in 19.05 is for evaluation only. > > > > > >
RE: [PATCH v2 5/8] vdpa/ifc: only configure enabled queue
> -Original Message-
> From: Xia, Chenbo
> Sent: Wednesday, September 14, 2022 09:59
> To: Pei, Andy ; dev@dpdk.org
> Cc: Xu, Rosen ; Huang, Wei ; Cao,
> Gang ; maxime.coque...@redhat.com; Huang Wei
>
> Subject: RE: [PATCH v2 5/8] vdpa/ifc: only configure enabled queue
>
> > -Original Message-
> > From: Pei, Andy
> > Sent: Thursday, September 8, 2022 1:54 PM
> > To: dev@dpdk.org
> > Cc: Xia, Chenbo ; Xu, Rosen
> > ; Huang, Wei ; Cao, Gang
> > ; maxime.coque...@redhat.com; Huang Wei
> >
> > Subject: [PATCH v2 5/8] vdpa/ifc: only configure enabled queue
> >
> > when configure the hardware queue, we only configure queues which have
> > been enabled by vhost.
> >
> > Signed-off-by: Andy Pei
> > Signed-off-by: Huang Wei
> > ---
> > drivers/vdpa/ifc/base/ifcvf.c | 6 +-
> > drivers/vdpa/ifc/ifcvf_vdpa.c | 16 ++--
> > 2 files changed, 19 insertions(+), 3 deletions(-)
> >
> > diff --git a/drivers/vdpa/ifc/base/ifcvf.c
> > b/drivers/vdpa/ifc/base/ifcvf.c index 0444d74..4875ea1 100644
> > --- a/drivers/vdpa/ifc/base/ifcvf.c
> > +++ b/drivers/vdpa/ifc/base/ifcvf.c
> > @@ -249,6 +249,9 @@
> >
> > ifcvf_enable_multiqueue(hw);
> > for (i = 0; i < hw->nr_vring; i++) {
> > + if (!hw->vring[i].enable)
> > + continue;
> > +
> > IFCVF_WRITE_REG16(i, &cfg->queue_select);
> > io_write64_twopart(hw->vring[i].desc, &cfg->queue_desc_lo,
> > &cfg->queue_desc_hi);
> > @@ -283,7 +286,8 @@
> > notify_off = IFCVF_READ_REG16(&cfg->queue_notify_off);
> > hw->notify_addr[i] = (void *)((u8 *)hw->notify_base +
> > notify_off * hw->notify_off_multiplier);
> > - IFCVF_WRITE_REG16(1, &cfg->queue_enable);
> > + if (hw->vring[i].enable)
>
> Seems useless check as it already checked when the for loop starts
Yes, it should be removed.
>
> Thanks,
> Chenbo
>
> > + IFCVF_WRITE_REG16(1, &cfg->queue_enable);
> > }
> >
> > return 0;
> > diff --git a/drivers/vdpa/ifc/ifcvf_vdpa.c
> > b/drivers/vdpa/ifc/ifcvf_vdpa.c index 2b42850..48f1a89 100644
> > --- a/drivers/vdpa/ifc/ifcvf_vdpa.c
> > +++ b/drivers/vdpa/ifc/ifcvf_vdpa.c
> > @@ -290,6 +290,8 @@ struct rte_vdpa_dev_info {
> > rte_vhost_get_negotiated_features(vid, &hw->req_features);
> >
> > for (i = 0; i < nr_vring; i++) {
> > + if (!hw->vring[i].enable)
> > + continue;
> > rte_vhost_get_vhost_vring(vid, i, &vq);
> > gpa = hva_to_gpa(vid, (uint64_t)(uintptr_t)vq.desc);
> > if (gpa == 0) {
> > @@ -505,6 +507,8 @@ struct rte_vdpa_dev_info {
> >
> > vring.kickfd = -1;
> > for (qid = 0; qid < q_num; qid++) {
> > + if (!hw->vring[qid].enable)
> > + continue;
> > ev.events = EPOLLIN | EPOLLPRI;
> > rte_vhost_get_vhost_vring(internal->vid, qid, &vring);
> > ev.data.u64 = qid | (uint64_t)vring.kickfd << 32; @@ -1064,6
> > +1068,8 @@ struct rte_vdpa_dev_info {
> > struct rte_vdpa_device *vdev;
> > struct internal_list *list;
> > struct ifcvf_internal *internal;
> > + struct ifcvf_hw *hw;
> > + uint16_t i;
> >
> > vdev = rte_vhost_get_vdpa_device(vid);
> > list = find_internal_resource_by_vdev(vdev);
> > @@ -1077,11 +1083,17 @@ struct rte_vdpa_dev_info {
> > rte_atomic32_set(&internal->dev_attached, 1);
> > update_datapath(internal);
> >
> > - if (rte_vhost_host_notifier_ctrl(vid, RTE_VHOST_QUEUE_ALL, true) !=
> > 0)
> > - DRV_LOG(NOTICE, "vDPA (%s): software relay is used.",
> > + hw = &internal->hw;
> > + for (i = 0; i < hw->nr_vring; i++) {
> > + if (!hw->vring[i].enable)
> > + continue;
> > + if (rte_vhost_host_notifier_ctrl(vid, i, true) != 0)
> > + DRV_LOG(NOTICE, "vDPA (%s): software relay is used.",
> > vdev->device->name);
> > + }
> >
> > internal->configured = 1;
> > + DRV_LOG(INFO, "vDPA device %s is configured", vdev->device->name);
> > return 0;
> > }
> >
> > --
> > 1.8.3.1
RE: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queue
> -Original Message-
> From: Xia, Chenbo
> Sent: Wednesday, September 14, 2022 10:23
> To: Pei, Andy ; dev@dpdk.org
> Cc: Xu, Rosen ; Huang, Wei ; Cao,
> Gang ; maxime.coque...@redhat.com; Huang Wei
>
> Subject: RE: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queue
>
> > -Original Message-
> > From: Pei, Andy
> > Sent: Thursday, September 8, 2022 1:54 PM
> > To: dev@dpdk.org
> > Cc: Xia, Chenbo ; Xu, Rosen
> > ; Huang, Wei ; Cao, Gang
> > ; maxime.coque...@redhat.com; Huang Wei
> >
> > Subject: [PATCH v2 6/8] vdpa/ifc: support dynamic enable/disable queue
> >
> > From: Huang Wei
> >
> > Support dynamic enable or disable queue.
> > For front end, like QEMU, user can use ethtool to configurate queue.
> > For example, "ethtool -L eth0 combined 3" to enable 3 queues pairs.
> >
> > Signed-off-by: Huang Wei
> > Signed-off-by: Andy Pei
> > ---
> > drivers/vdpa/ifc/base/ifcvf.c | 101
> >
> > drivers/vdpa/ifc/base/ifcvf.h | 6 +++
> > drivers/vdpa/ifc/base/ifcvf_osdep.h | 1 +
> > drivers/vdpa/ifc/ifcvf_vdpa.c | 93 +++
> > --
> > 4 files changed, 186 insertions(+), 15 deletions(-)
> >
> > diff --git a/drivers/vdpa/ifc/base/ifcvf.c
> > b/drivers/vdpa/ifc/base/ifcvf.c index 4875ea1..34f32f8 100644
> > --- a/drivers/vdpa/ifc/base/ifcvf.c
> > +++ b/drivers/vdpa/ifc/base/ifcvf.c
> > @@ -230,6 +230,107 @@
> > }
> > }
> >
> > +int
> > +ifcvf_enable_vring_hw(struct ifcvf_hw *hw, int i) {
> > + struct ifcvf_pci_common_cfg *cfg;
> > + u8 *lm_cfg;
> > + u16 notify_off;
> > + int msix_vector;
> > +
> > + if (!hw || (i >= (int)hw->nr_vring))
> > + return -1;
>
> Seems HW will always be not NULL
As a external function, we should not assume the input argument is always valid.
>
> > +
> > + cfg = hw->common_cfg;
> > + if (!cfg) {
> > + ERROUT("common_cfg in HW is NULL.\n");
>
> I am thinking why you introduce this new log? Why not just use DRV_LOG that is
> already defined?
Because below type of log macros are already defined and used in original code,
I just follow its tradition.
#define WARNINGOUT(S, args...) RTE_LOG(WARNING, PMD, S, ##args)
#define DEBUGOUT(S, args...)RTE_LOG(DEBUG, PMD, S, ##args)
#define ERROUT(S, args...) RTE_LOG(ERR, PMD, S, ##args)
>
> > + return -1;
> > + }
> > +
> > + ifcvf_enable_multiqueue(hw);
> > +
> > + IFCVF_WRITE_REG16(i, &cfg->queue_select);
> > + msix_vector = IFCVF_READ_REG16(&cfg->queue_msix_vector);
> > + if (msix_vector != (i + 1)) {
> > + IFCVF_WRITE_REG16(i + 1, &cfg->queue_msix_vector);
> > + msix_vector = IFCVF_READ_REG16(&cfg->queue_msix_vector);
> > + if (msix_vector == IFCVF_MSI_NO_VECTOR) {
> > + ERROUT("queue %u, msix vec alloc failed\n", i);
> > + return -1;
> > + }
> > + }
> > +
> > + io_write64_twopart(hw->vring[i].desc, &cfg->queue_desc_lo,
> > + &cfg->queue_desc_hi);
> > + io_write64_twopart(hw->vring[i].avail, &cfg->queue_avail_lo,
> > + &cfg->queue_avail_hi);
> > + io_write64_twopart(hw->vring[i].used, &cfg->queue_used_lo,
> > + &cfg->queue_used_hi);
> > + IFCVF_WRITE_REG16(hw->vring[i].size, &cfg->queue_size);
> > +
> > + lm_cfg = hw->lm_cfg;
> > + if (lm_cfg) {
> > + if (hw->device_type == IFCVF_BLK)
> > + *(u32 *)(lm_cfg + IFCVF_LM_RING_STATE_OFFSET +
> > + i * IFCVF_LM_CFG_SIZE) =
> > + (u32)hw->vring[i].last_avail_idx |
> > + ((u32)hw->vring[i].last_used_idx << 16);
> > + else
> > + *(u32 *)(lm_cfg + IFCVF_LM_RING_STATE_OFFSET +
> > + (i / 2) * IFCVF_LM_CFG_SIZE +
> > + (i % 2) * 4) =
> > + (u32)hw->vring[i].last_avail_idx |
> > + ((u32)hw->vring[i].last_used_idx << 16);
> > + }
>
> So the register layout is different for blk and net?
That's sure.
>
> > +
> > + notify_off = IFCVF_READ_REG16(&cfg->queue_notify_off);
> > + hw->notify_add
Re: [dpdk-dev] [dpdk-stable] [PATCH v16 0/3] raw/ifpga: add extra APIs for Cyborg
Ferruh, Yes, they have been rejected. Thanks -Original Message- From: Yigit, Ferruh Sent: Thursday, March 18, 2021 01:09 To: Huang, Wei ; dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z Cc: sta...@dpdk.org; Zhang, Tianfei Subject: Re: [dpdk-stable] [PATCH v16 0/3] raw/ifpga: add extra APIs for Cyborg On 3/17/2021 7:59 AM, Wei Huang wrote: > Cyborg is part of OpenStack, it needs some extra APIs to manage > devices with Intel FPGA. These patches add APIs to meet Cyborg > requirement. > > Main changes from v15: > - remove example from the patch set > > Wei Huang (3): >raw/ifpga: add fpga rsu APIs >raw/ifpga: add APIs to get fpga information >raw/ifpga: add miscellaneous APIs > I guess this is sent by mistake, v16 already merged, updating patchwork accordingly.
RE: [PATCH v1] raw/ifpga: fix interrupt handle allocation
Hi Thomas,
> -Original Message-
> From: Thomas Monjalon
> Sent: Tuesday, March 8, 2022 06:24
> To: Huang, Wei ; Zhang, Tianfei
>
> Cc: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z
> ; nipun.gu...@nxp.com; hemant.agra...@nxp.com;
> sta...@dpdk.org; Yigit, Ferruh
> Subject: Re: [PATCH v1] raw/ifpga: fix interrupt handle allocation
>
> 21/02/2022 07:51, Zhang, Tianfei:
> > From: Huang, Wei
> > > Allocate FPGA interrupt handle instance for each card.
>
> It doesn't say what was the issue.
Original code use global interrupt handle for all cards, the same interrupt
handle will be registered by each card, so only the last card will be
interrupted correctly,
it's the problem.
>
> > >
> > > Fixes: e0a1aafe2af9 ("raw/ifpga: introduce IRQ functions")
> > > Cc: sta...@dpdk.org
> > >
> > > Signed-off-by: Wei Huang
> >
> > It looks good for me.
> >
> > Acked-by: Tianfei Zhang
>
> Applied
>
RE: [PATCH v1] raw/ifpga: fix monitor thread issues
Hi Thomas,
> -Original Message-
> From: Thomas Monjalon
> Sent: Tuesday, March 8, 2022 06:30
> To: Zhang, Tianfei ; Huang, Wei
>
> Cc: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z
> ; nipun.gu...@nxp.com; hemant.agra...@nxp.com;
> sta...@dpdk.org; Yigit, Ferruh
> Subject: Re: [PATCH v1] raw/ifpga: fix monitor thread issues
>
> 01/03/2022 09:47, Wei Huang:
> > Monitor thread handles graceful shutdown according to the value of
> > specific sensors in device, two issues are found below.
> > 1. Thread is not created when card is probed.
> > 2. Thread is canceled without checking presence of other cards.
> > To fix them, thread is created in pci device probe function, a
> > reference count is checked before canceling the thread.
> >
> > Fixes: 9c006c45 ("raw/ifpga: scan PCIe BDF device tree")
> > Cc: sta...@dpdk.org
> >
> > Signed-off-by: Wei Huang
> > Acked-by: Tianfei Zhang
>
> In general, it is better to have 2 separate patches for 2 different issues.
>
These two issue are related, if I split them into two patches, the thread
cannot work correctly with any one of the patches applied.
> Applied
>
>
RE: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
Hi Thomas,
Would you please check this patch is acceptable ?
> -Original Message-
> From: Huang, Wei
> Sent: Friday, March 18, 2022 16:58
> To: dev@dpdk.org; tho...@monjalon.net; Xu, Rosen
> ; nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Zhang, Qi Z
> ; Yigit, Ferruh ; Huang, Wei
>
> Subject: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
>
> Virtual devices created on ifpga raw device are not removed when ifpga is
> closed. To avoid such problem, ifpga virtual device remove function is
> implemented, virtual device is removed in raw device close function.
>
> Signed-off-by: Wei Huang
> Acked-by: Tianfei Zhang
> Acked-by: Rosen Xu
> ---
> v2: cleanup vdev_name no matter rte_vdev_uninit succeed or not
> ---
> v3: cleanup vdev_name in ifpga_cfg_remove
> ---
> drivers/raw/ifpga/ifpga_rawdev.c | 166
> ++-
> drivers/raw/ifpga/ifpga_rawdev.h | 8 ++
> 2 files changed, 138 insertions(+), 36 deletions(-)
>
> diff --git a/drivers/raw/ifpga/ifpga_rawdev.c
> b/drivers/raw/ifpga/ifpga_rawdev.c
> index 6d4117c..fe3fc43 100644
> --- a/drivers/raw/ifpga/ifpga_rawdev.c
> +++ b/drivers/raw/ifpga/ifpga_rawdev.c
> @@ -134,6 +134,8 @@ struct ifpga_rawdev *
> for (i = 0; i < IFPGA_MAX_IRQ; i++)
> dev->intr_handle[i] = NULL;
> dev->poll_enabled = 0;
> + for (i = 0; i < IFPGA_MAX_VDEV; i++)
> + dev->vdev_name[i] = NULL;
>
> return dev;
> }
> @@ -736,10 +738,22 @@ static int set_surprise_link_check_aer( static int
> ifpga_rawdev_close(struct rte_rawdev *dev) {
> + struct ifpga_rawdev *ifpga_rdev = NULL;
> struct opae_adapter *adapter;
> + char *vdev_name = NULL;
> + int i = 0;
>
> if (dev) {
> - ifpga_monitor_stop_func(ifpga_rawdev_get(dev));
> + ifpga_rdev = ifpga_rawdev_get(dev);
> + if (ifpga_rdev) {
> + for (i = 0; i < IFPGA_MAX_VDEV; i++) {
> + vdev_name = ifpga_rdev->vdev_name[i];
> + if (vdev_name)
> + rte_vdev_uninit(vdev_name);
> + }
> + ifpga_monitor_stop_func(ifpga_rdev);
> + ifpga_rdev->rawdev = NULL;
> + }
> adapter = ifpga_rawdev_get_priv(dev);
> if (adapter) {
> opae_adapter_destroy(adapter);
> @@ -1638,8 +1652,6 @@ static int fme_clean_fme_error(struct
> opae_manager *mgr)
> return -EINVAL;
> }
> dev = ifpga_rawdev_get(rawdev);
> - if (dev)
> - dev->rawdev = NULL;
>
> adapter = ifpga_rawdev_get_priv(rawdev);
> if (!adapter)
> @@ -1714,73 +1726,118 @@ static int ifpga_rawdev_get_string_arg(const
> char *key __rte_unused,
>
> return 0;
> }
> +
> static int
> -ifpga_cfg_probe(struct rte_vdev_device *dev)
> +ifpga_vdev_parse_devargs(struct rte_devargs *devargs,
> + struct ifpga_vdev_args *args)
> {
> - struct rte_devargs *devargs;
> - struct rte_kvargs *kvlist = NULL;
> - struct rte_rawdev *rawdev = NULL;
> - struct ifpga_rawdev *ifpga_dev;
> - int port;
> + struct rte_kvargs *kvlist;
> char *name = NULL;
> - char dev_name[RTE_RAWDEV_NAME_MAX_LEN];
> - int ret = -1;
> + int port = 0;
> + int ret = -EINVAL;
>
> - devargs = dev->device.devargs;
> + if (!devargs || !args)
> + return ret;
>
> kvlist = rte_kvargs_parse(devargs->args, valid_args);
> if (!kvlist) {
> - IFPGA_RAWDEV_PMD_LOG(ERR, "error when parsing
> param");
> - goto end;
> + IFPGA_RAWDEV_PMD_ERR("error when parsing devargs");
> + return ret;
> }
>
> if (rte_kvargs_count(kvlist, IFPGA_ARG_NAME) == 1) {
> if (rte_kvargs_process(kvlist, IFPGA_ARG_NAME,
> -&ifpga_rawdev_get_string_arg,
> -&name) < 0) {
> + &ifpga_rawdev_get_string_arg, &name) < 0) {
> IFPGA_RAWDEV_PMD_ERR("error to parse %s",
> - IFPGA_ARG_NAME);
> + IFPGA_ARG_NAME);
> goto end;
> + } else {
> + strlcpy(args->bdf, name, sizeof(args->bdf));
> + rte_free(name);
> }
> } else {
> IFPGA_RAWDEV_PMD_ERR(
RE: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
Got it, thanks.
> -Original Message-
> From: Zhang, Qi Z
> Sent: Tuesday, March 29, 2022 10:15
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; Xu, Rosen ;
> nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Yigit, Ferruh
>
> Subject: RE: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Tuesday, March 29, 2022 9:56 AM
> > To: dev@dpdk.org; tho...@monjalon.net; Xu, Rosen
> ;
> > nipun.gu...@nxp.com; hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Zhang, Tianfei ; Zhang,
> > Qi Z ; Yigit, Ferruh
> > Subject: RE: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
> >
> > Hi Thomas,
> >
> > Would you please check this patch is acceptable ?
>
> Hi Wei:
> 22.07 cycle just begins, It may not necessary to push PMD merge quite early
> ๐
> Below is the roadmap for your reference
>
> http://core.dpdk.org/roadmap/
>
> Proposal deadline (RFC/v1 patches): 10 April 2022 API freeze (-rc1): 30 May
> 2022 PMD features freeze (-rc2): 20 June 2022 Builtin applications features
> freeze (-rc3): 27 June 2022
> Release: 13 July 2022
>
> Regards
> Qi
>
> >
> > > -Original Message-
> > > From: Huang, Wei
> > > Sent: Friday, March 18, 2022 16:58
> > > To: dev@dpdk.org; tho...@monjalon.net; Xu, Rosen
> > ;
> > > nipun.gu...@nxp.com; hemant.agra...@nxp.com
> > > Cc: sta...@dpdk.org; Zhang, Tianfei ;
> > > Zhang, Qi Z ; Yigit, Ferruh
> > > ; Huang, Wei
> > > Subject: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
> > >
> > > Virtual devices created on ifpga raw device are not removed when
> > > ifpga is closed. To avoid such problem, ifpga virtual device remove
> > > function is implemented, virtual device is removed in raw device close
> function.
> > >
> > > Signed-off-by: Wei Huang
> > > Acked-by: Tianfei Zhang
> > > Acked-by: Rosen Xu
> > > ---
> > > v2: cleanup vdev_name no matter rte_vdev_uninit succeed or not
> > > ---
> > > v3: cleanup vdev_name in ifpga_cfg_remove
> > > ---
> > > drivers/raw/ifpga/ifpga_rawdev.c | 166
> > > ++-
> > > drivers/raw/ifpga/ifpga_rawdev.h | 8 ++
> > > 2 files changed, 138 insertions(+), 36 deletions(-)
> > >
> > > diff --git a/drivers/raw/ifpga/ifpga_rawdev.c
> > > b/drivers/raw/ifpga/ifpga_rawdev.c
> > > index 6d4117c..fe3fc43 100644
> > > --- a/drivers/raw/ifpga/ifpga_rawdev.c
> > > +++ b/drivers/raw/ifpga/ifpga_rawdev.c
> > > @@ -134,6 +134,8 @@ struct ifpga_rawdev * for (i = 0; i <
> > > IFPGA_MAX_IRQ; i++) dev->intr_handle[i] = NULL; dev->poll_enabled
> > > = 0;
> > > +for (i = 0; i < IFPGA_MAX_VDEV; i++)
> > > +dev->vdev_name[i] = NULL;
> > >
> > > return dev;
> > > }
> > > @@ -736,10 +738,22 @@ static int set_surprise_link_check_aer(
> > > static int ifpga_rawdev_close(struct rte_rawdev *dev) {
> > > +struct ifpga_rawdev *ifpga_rdev = NULL;
> > > struct opae_adapter *adapter;
> > > +char *vdev_name = NULL;
> > > +int i = 0;
> > >
> > > if (dev) {
> > > -ifpga_monitor_stop_func(ifpga_rawdev_get(dev));
> > > +ifpga_rdev = ifpga_rawdev_get(dev); if (ifpga_rdev) { for (i = 0; i
> > > +< IFPGA_MAX_VDEV; i++) { vdev_name = ifpga_rdev->vdev_name[i]; if
> > > +(vdev_name) rte_vdev_uninit(vdev_name); }
> > > +ifpga_monitor_stop_func(ifpga_rdev);
> > > +ifpga_rdev->rawdev = NULL;
> > > +}
> > > adapter = ifpga_rawdev_get_priv(dev); if (adapter) {
> > > opae_adapter_destroy(adapter); @@ -1638,8 +1652,6 @@ static int
> > > fme_clean_fme_error(struct opae_manager *mgr) return -EINVAL; }
> > > dev = ifpga_rawdev_get(rawdev); -if (dev)
> > > -dev->rawdev = NULL;
> > >
> > > adapter = ifpga_rawdev_get_priv(rawdev); if (!adapter) @@ -1714,73
> > > +1726,118 @@ static int ifpga_rawdev_get_string_arg(const char *key
> > > __rte_unused,
> > >
> > > return 0;
> > > }
> > > +
> > > static int
> > > -ifpga_cfg_probe(struct rte_vdev_device *dev)
> > > +ifpga_vdev_parse_devargs(struct rte_devargs *devargs, struct
> > > +ifpga_vdev_args *args)
> > > {
> > > -struct rte_devargs *devargs;
> > > -struct rte_kvargs *kvlist = NULL;
> > > -str
RE: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
> -Original Message-
> From: Zhang, Qi Z
> Sent: Tuesday, March 29, 2022 10:15
> To: Huang, Wei ; dev@dpdk.org;
> tho...@monjalon.net; Xu, Rosen ;
> nipun.gu...@nxp.com; hemant.agra...@nxp.com
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Yigit, Ferruh
>
> Subject: RE: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
>
>
> > -----Original Message-
> > From: Huang, Wei
> > Sent: Tuesday, March 29, 2022 9:56 AM
> > To: dev@dpdk.org; tho...@monjalon.net; Xu, Rosen
> ;
> > nipun.gu...@nxp.com; hemant.agra...@nxp.com
> > Cc: sta...@dpdk.org; Zhang, Tianfei ; Zhang,
> > Qi Z ; Yigit, Ferruh
> > Subject: RE: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
> >
> > Hi Thomas,
> >
> > Would you please check this patch is acceptable ?
>
> Hi Wei:
> 22.07 cycle just begins, It may not necessary to push PMD merge quite early
> ๐
> Below is the roadmap for your reference
>
> http://core.dpdk.org/roadmap/
>
> Proposal deadline (RFC/v1 patches): 10 April 2022 API freeze (-rc1): 30 May
> 2022 PMD features freeze (-rc2): 20 June 2022 Builtin applications features
> freeze (-rc3): 27 June 2022
> Release: 13 July 2022
>
> Regards
> Qi
>
Hi Qi and Thomas,
This patch is not new feature, it's a fix, can it be merged now ?
> >
> > > -Original Message-
> > > From: Huang, Wei
> > > Sent: Friday, March 18, 2022 16:58
> > > To: dev@dpdk.org; tho...@monjalon.net; Xu, Rosen
> > ;
> > > nipun.gu...@nxp.com; hemant.agra...@nxp.com
> > > Cc: sta...@dpdk.org; Zhang, Tianfei ;
> > > Zhang, Qi Z ; Yigit, Ferruh
> > > ; Huang, Wei
> > > Subject: [PATCH v3] raw/ifpga: remove vdev when ifpga is closed
> > >
> > > Virtual devices created on ifpga raw device are not removed when
> > > ifpga is closed. To avoid such problem, ifpga virtual device remove
> > > function is implemented, virtual device is removed in raw device close
> function.
> > >
> > > Signed-off-by: Wei Huang
> > > Acked-by: Tianfei Zhang
> > > Acked-by: Rosen Xu
> > > ---
> > > v2: cleanup vdev_name no matter rte_vdev_uninit succeed or not
> > > ---
> > > v3: cleanup vdev_name in ifpga_cfg_remove
> > > ---
> > > drivers/raw/ifpga/ifpga_rawdev.c | 166
> > > ++-
> > > drivers/raw/ifpga/ifpga_rawdev.h | 8 ++
> > > 2 files changed, 138 insertions(+), 36 deletions(-)
> > >
> > > diff --git a/drivers/raw/ifpga/ifpga_rawdev.c
> > > b/drivers/raw/ifpga/ifpga_rawdev.c
> > > index 6d4117c..fe3fc43 100644
> > > --- a/drivers/raw/ifpga/ifpga_rawdev.c
> > > +++ b/drivers/raw/ifpga/ifpga_rawdev.c
> > > @@ -134,6 +134,8 @@ struct ifpga_rawdev * for (i = 0; i <
> > > IFPGA_MAX_IRQ; i++) dev->intr_handle[i] = NULL; dev->poll_enabled
> > > = 0;
> > > +for (i = 0; i < IFPGA_MAX_VDEV; i++)
> > > +dev->vdev_name[i] = NULL;
> > >
> > > return dev;
> > > }
> > > @@ -736,10 +738,22 @@ static int set_surprise_link_check_aer(
> > > static int ifpga_rawdev_close(struct rte_rawdev *dev) {
> > > +struct ifpga_rawdev *ifpga_rdev = NULL;
> > > struct opae_adapter *adapter;
> > > +char *vdev_name = NULL;
> > > +int i = 0;
> > >
> > > if (dev) {
> > > -ifpga_monitor_stop_func(ifpga_rawdev_get(dev));
> > > +ifpga_rdev = ifpga_rawdev_get(dev); if (ifpga_rdev) { for (i = 0; i
> > > +< IFPGA_MAX_VDEV; i++) { vdev_name = ifpga_rdev->vdev_name[i]; if
> > > +(vdev_name) rte_vdev_uninit(vdev_name); }
> > > +ifpga_monitor_stop_func(ifpga_rdev);
> > > +ifpga_rdev->rawdev = NULL;
> > > +}
> > > adapter = ifpga_rawdev_get_priv(dev); if (adapter) {
> > > opae_adapter_destroy(adapter); @@ -1638,8 +1652,6 @@ static int
> > > fme_clean_fme_error(struct opae_manager *mgr) return -EINVAL; }
> > > dev = ifpga_rawdev_get(rawdev); -if (dev)
> > > -dev->rawdev = NULL;
> > >
> > > adapter = ifpga_rawdev_get_priv(rawdev); if (!adapter) @@ -1714,73
> > > +1726,118 @@ static int ifpga_rawdev_get_string_arg(const char *key
> > > __rte_unused,
> > >
> > > return 0;
> > > }
> > > +
> > > static int
> > > -ifpga_cfg_probe(struct rte_vdev_device *dev)
> > > +ifpga_vdev_parse_devargs(struct rte_devargs *devargs, struct
> > > +ifpga_vdev_args *args)
> > > {
> >
Re: [dpdk-dev] [PATCH v4 3/4] raw/ifpga: add opae API for Cyborg
Hi Rosen,
I cannot change "master" to "primary", how to fix it ?
Wei
-Original Message-
From: Xu, Rosen
Sent: Friday, January 1, 2021 22:39
To: Huang, Wei ; dev@dpdk.org; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei
Subject: RE: [PATCH v4 3/4] raw/ifpga: add opae API for Cyborg
Hi Wei,
Could you pls fix coding style issues? Thanks a lot.
Thanks,
Rosen
> -Original Message-
> From: Huang, Wei
> Sent: Wednesday, December 30, 2020 15:46
> To: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Huang, Wei
>
> Subject: [PATCH v4 3/4] raw/ifpga: add opae API for Cyborg
>
> Cyborg is part of OpenStack, it needs some OPAE type APIs to manage
> PACs (Programmable Acceleration Card) with Intel FPGA. Below major
> functions are added to meets Cyborg requirements.
> 1. opae_init_eal() set up EAL environment.
> 2. opae_cleanup_eal() clean up EAL environment.
> 3. opae_enumerate() searches PAC with specific FPGA.
> 4. opae_get_property() gets properties of FPGA.
> 5. opae_partial_reconfigure() perform partial configuration on FPGA.
> 6. opae_get_image_info() gets information of image file.
> 7. opae_update_flash() updates FPGA flash with specific image file.
> 8. opae_cancel_flash_update() cancel process of FPGA flash update.
> 9. opae_probe_device() manually probe specific FPGA with ifpga driver.
> 10. opae_remove_device() manually remove specific FPGA from ifpga driver.
> 11. opae_bind_driver() binds specific FPGA with specified kernel driver.
> 12. opae_unbind_driver() unbinds specific FPGA from kernel driver.
> 13. opae_reboot_device() reboots specific FPGA (do reconfiguration).
>
> Signed-off-by: Wei Huang
> ---
> v2: fix typo in commit log and ifpga_opae_api.h
> ---
> v3: fix coding style issue in ifpga_opae_api.c
> ---
> v4: enclose macro PCI_EXT_CAP_ID in parentheses
> ---
> drivers/raw/ifpga/ifpga_opae_api.c | 1801
>
> drivers/raw/ifpga/ifpga_opae_api.h | 245
> drivers/raw/ifpga/ifpga_rawdev.c | 152 ++-
> drivers/raw/ifpga/ifpga_rawdev.h | 15 +
> drivers/raw/ifpga/meson.build |4 +-
> 5 files changed, 2210 insertions(+), 7 deletions(-)
> create mode 100644 drivers/raw/ifpga/ifpga_opae_api.c
> create mode 100644 drivers/raw/ifpga/ifpga_opae_api.h
>
> diff --git a/drivers/raw/ifpga/ifpga_opae_api.c
> b/drivers/raw/ifpga/ifpga_opae_api.c
> new file mode 100644
> index 0..9210092ab
> --- /dev/null
> +++ b/drivers/raw/ifpga/ifpga_opae_api.c
> @@ -0,0 +1,1801 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2020 Intel Corporation
> + */
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include "base/opae_hw_api.h"
> +#include "base/ifpga_sec_mgr.h"
> +#include "ifpga_rawdev.h"
> +#include "ifpga_opae_api.h"
> +
> +
> +int opae_log_level;
> +FILE *opae_log_file;
> +
> +static opae_api_version api_ver = {21, 2, 0};
> +static int eal_inited;
> +static uint32_t dev_aer[2] = {0};
> +
> +static const char * const log_level_name[] = {"CRITICAL", "ERROR",
> + "WARNING", "INFORMATION", "DEBUG"};
> +static const char * const proc_type_name[] = {"NON-DPDK", "PRIMARY",
> + "SECONDARY"};
> +static const char * const platform_name[] = {"Vista Creek", "Rush Creek",
> + "Darby Creek", "Lightning Creek"};
> +static const char * const release_name[] = {"Pre-Alpha", "Alpha", "Beta",
> "PV"};
> +static const char * const interface_type[] = {"8x10G", "4x25G", "2x1x25G",
> + "4x25G+2x25G", "2x2x25G", "2x1x25Gx2FVL", "1x2x25G"};
> +static const char * const kdrv[] = {OPAE_KDRV_UNKNOWN,
> OPAE_KDRV_IGB_UIO,
> + OPAE_KDRV_VFIO_PCI, OPAE_KDRV_UIO_PCI};
> +
> +RTE_INIT(init_api_env)
> +{
> + eal_inited = 0;
> + opae_log_level = OPAE_LOG_ERR;
> + opae_log_file = NULL;
> + ifpga_rawdev_logtype = 0;
> +
> + opae_log_info("API environment is initialized\n");
> +}
> +
> +RTE_FINI(clean_api_env)
> +{
> + if (opae_log_file) {
> + fclose(opae_log_file);
> + opae_log_file = NULL;
> + }
> + opae_log_info("API environment is cleaned\n");
> +}
> +
> +void opae_get_api_version(opae_api_version *version)
> +{
> + if (version)
> + memcpy(version, &api_ver, sizeof(opae_api_version)
Re: [dpdk-dev] [PATCH v9 1/4] raw/ifpga: add fpga rsu function
Hi,
-Original Message-
From: Xu, Rosen
Sent: Tuesday, January 12, 2021 18:46
To: Huang, Wei ; dev@dpdk.org; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei
Subject: RE: [PATCH v9 1/4] raw/ifpga: add fpga rsu function
Hi,
> -Original Message-
> From: Huang, Wei
> Sent: Tuesday, January 12, 2021 13:18
> To: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Huang,
> Wei
> Subject: [PATCH v9 1/4] raw/ifpga: add fpga rsu function
>
> RSU (Remote System Update) depends on secure manager which may be
> different on various implementations, so a new secure manager device
> is implemented for adapting such difference.
> There are three major functions added:
> 1. ifpga_rawdev_update_flash() updates flash with specific image file.
> 2. ifpga_rawdev_stop_flash_update() aborts flash update process.
> 3. ifpga_rawdev_reload() reloads FPGA from updated flash.
>
> Signed-off-by: Wei Huang
> ---
> v2: fix coding style issue in ifpga_fme_rsu.c and ifpga_sec_mgr.c
> ---
> v3: fix compilation issues in ifpga_fme_rsu.c
> ---
> v4: fix compilation issues in opae_intel_max10.c
> ---
> drivers/raw/ifpga/base/ifpga_api.c| 26 +
> drivers/raw/ifpga/base/ifpga_fme.c| 8 +
> drivers/raw/ifpga/base/ifpga_fme_rsu.c| 435 +++
> drivers/raw/ifpga/base/ifpga_hw.h | 1 +
> drivers/raw/ifpga/base/ifpga_sec_mgr.c| 639 ++
> drivers/raw/ifpga/base/ifpga_sec_mgr.h| 89 +++
> drivers/raw/ifpga/base/meson.build| 2 +
> drivers/raw/ifpga/base/opae_hw_api.c | 59 ++
> drivers/raw/ifpga/base/opae_hw_api.h | 11 +
> drivers/raw/ifpga/base/opae_intel_max10.c | 48 ++
> drivers/raw/ifpga/base/opae_intel_max10.h | 44 ++
> drivers/raw/ifpga/ifpga_rawdev.c | 55 ++
> drivers/raw/ifpga/ifpga_rawdev.h | 7 +-
> 13 files changed, 1423 insertions(+), 1 deletion(-) create mode
> 100644 drivers/raw/ifpga/base/ifpga_fme_rsu.c
> create mode 100644 drivers/raw/ifpga/base/ifpga_sec_mgr.c
> create mode 100644 drivers/raw/ifpga/base/ifpga_sec_mgr.h
>
> diff --git a/drivers/raw/ifpga/base/ifpga_api.c
> b/drivers/raw/ifpga/base/ifpga_api.c
> index 1ff57fa18..1aedf150b 100644
> --- a/drivers/raw/ifpga/base/ifpga_api.c
> +++ b/drivers/raw/ifpga/base/ifpga_api.c
> @@ -5,6 +5,7 @@
> #include "ifpga_api.h"
> #include "ifpga_enumerate.h"
> #include "ifpga_feature_dev.h"
> +#include "ifpga_sec_mgr.h"
>
> #include "opae_hw_api.h"
>
> @@ -228,11 +229,36 @@ static int ifpga_mgr_get_board_info(struct
> opae_manager *mgr,
> return 0;
> }
>
> +static int ifpga_mgr_update_flash(struct opae_manager *mgr, const
> +char
> *image,
> + u64 *status)
> +{
> + struct ifpga_fme_hw *fme = mgr->data;
> +
> + return fpga_update_flash(fme, image, status); }
> +
> +static int ifpga_mgr_stop_flash_update(struct opae_manager *mgr, int
> +force) {
> + struct ifpga_fme_hw *fme = mgr->data;
> +
> + return fpga_stop_flash_update(fme, force); }
> +
> +static int ifpga_mgr_reload(struct opae_manager *mgr, int type, int
> +page) {
> + struct ifpga_fme_hw *fme = mgr->data;
> +
> + return fpga_reload(fme, type, page); }
> +
> struct opae_manager_ops ifpga_mgr_ops = {
> .flash = ifpga_mgr_flash,
> .get_eth_group_region_info = ifpga_mgr_get_eth_group_region_info,
> .get_sensor_value = ifpga_mgr_get_sensor_value,
> .get_board_info = ifpga_mgr_get_board_info,
> + .update_flash = ifpga_mgr_update_flash,
> + .stop_flash_update = ifpga_mgr_stop_flash_update,
> + .reload = ifpga_mgr_reload,
> };
>
> static int ifpga_mgr_read_mac_rom(struct opae_manager *mgr, int
> offset, diff --git a/drivers/raw/ifpga/base/ifpga_fme.c
> b/drivers/raw/ifpga/base/ifpga_fme.c
> index f29ff3159..34fd9a818 100644
> --- a/drivers/raw/ifpga/base/ifpga_fme.c
> +++ b/drivers/raw/ifpga/base/ifpga_fme.c
> @@ -7,6 +7,7 @@
> #include "opae_intel_max10.h"
> #include "opae_i2c.h"
> #include "opae_at24_eeprom.h"
> +#include "ifpga_sec_mgr.h"
>
> #define PWR_THRESHOLD_MAX 0x7F
>
> @@ -1152,6 +1153,12 @@ static int fme_nios_spi_init(struct
> ifpga_feature
> *feature)
> if (spi_self_checking(max10))
> goto spi_fail;
>
> + ret = init_sec_mgr(fme);
> + if (ret) {
> + dev_err(fme, "security manager init fail\n");
> + goto spi_fail;
> + }
> +
> return ret;
>
> spi_fail:
> @@ -1165,6 +1172,7 @@ static void fme_nios_spi_uinit(struct
> i
Re: [dpdk-dev] [PATCH v9 3/4] raw/ifpga: add opae API for Cyborg
Hi,
-Original Message-
From: Xu, Rosen
Sent: Tuesday, January 12, 2021 19:00
To: Huang, Wei ; dev@dpdk.org; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei
Subject: RE: [PATCH v9 3/4] raw/ifpga: add opae API for Cyborg
Hi,
> -Original Message-
> From: Huang, Wei
> Sent: Tuesday, January 12, 2021 13:18
> To: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Huang, Wei
>
> Subject: [PATCH v9 3/4] raw/ifpga: add opae API for Cyborg
>
> Cyborg is part of OpenStack, it needs some OPAE type APIs to manage
> PACs (Programmable Acceleration Card) with Intel FPGA. Below major
> functions are added to meets Cyborg requirements.
> 1. opae_init_eal() set up EAL environment.
> 2. opae_cleanup_eal() clean up EAL environment.
> 3. opae_enumerate() searches PAC with specific FPGA.
> 4. opae_get_property() gets properties of FPGA.
> 5. opae_partial_reconfigure() perform partial configuration on FPGA.
> 6. opae_get_image_info() gets information of image file.
> 7. opae_update_flash() updates FPGA flash with specific image file.
> 8. opae_cancel_flash_update() cancel process of FPGA flash update.
> 9. opae_probe_device() manually probe specific FPGA with ifpga driver.
> 10. opae_remove_device() manually remove specific FPGA from ifpga driver.
> 11. opae_bind_driver() binds specific FPGA with specified kernel driver.
> 12. opae_unbind_driver() unbinds specific FPGA from kernel driver.
> 13. opae_reboot_device() reboots specific FPGA (do reconfiguration).
>
> Signed-off-by: Wei Huang
> ---
> v2: fix typo in commit log and ifpga_opae_api.h
> ---
> v3: fix coding style issue in ifpga_opae_api.c
> ---
> v4: enclose macro PCI_EXT_CAP_ID in parentheses
> ---
> v5: fix icc compiler build error
> ---
> v6: fix compilation issues in ifpga_opae_api.c
> ---
> drivers/raw/ifpga/ifpga_opae_api.c | 1800
>
> drivers/raw/ifpga/ifpga_opae_api.h | 245
> drivers/raw/ifpga/ifpga_rawdev.c | 152 ++-
> drivers/raw/ifpga/ifpga_rawdev.h | 15 +
> drivers/raw/ifpga/meson.build |4 +-
> 5 files changed, 2209 insertions(+), 7 deletions(-)
> create mode 100644 drivers/raw/ifpga/ifpga_opae_api.c
> create mode 100644 drivers/raw/ifpga/ifpga_opae_api.h
>
> diff --git a/drivers/raw/ifpga/ifpga_opae_api.c
> b/drivers/raw/ifpga/ifpga_opae_api.c
> new file mode 100644
> index 0..42194058c
> --- /dev/null
> +++ b/drivers/raw/ifpga/ifpga_opae_api.c
> @@ -0,0 +1,1800 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2020 Intel Corporation
> + */
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include "base/opae_hw_api.h"
> +#include "base/ifpga_sec_mgr.h"
> +#include "ifpga_rawdev.h"
> +#include "ifpga_opae_api.h"
> +
> +
> +int opae_log_level;
> +FILE *opae_log_file;
> +
> +static opae_api_version api_ver = {21, 2, 0};
> +static int eal_inited;
> +static uint32_t dev_aer[2] = {0};
> +
> +static const char * const log_level_name[] = {"CRITICAL", "ERROR",
> + "WARNING", "INFORMATION", "DEBUG"};
> +static const char * const proc_type_name[] = {"NON-DPDK", "PRIMARY",
> + "SECONDARY"};
> +static const char * const platform_name[] = {"Vista Creek", "Rush Creek",
> + "Darby Creek", "Lightning Creek"};
> +static const char * const release_name[] = {"Pre-Alpha", "Alpha", "Beta",
> "PV"};
> +static const char * const interface_type[] = {"8x10G", "4x25G", "2x1x25G",
> + "4x25G+2x25G", "2x2x25G", "2x1x25Gx2FVL", "1x2x25G"};
> +static const char * const kdrv[] = {OPAE_KDRV_UNKNOWN,
> OPAE_KDRV_IGB_UIO,
> + OPAE_KDRV_VFIO_PCI, OPAE_KDRV_UIO_PCI};
> +
> +RTE_INIT(init_api_env)
> +{
> + eal_inited = 0;
> + opae_log_level = OPAE_LOG_ERR;
> + opae_log_file = NULL;
> + ifpga_rawdev_logtype = 0;
> +
> + opae_log_info("API environment is initialized\n");
> +}
> +
> +RTE_FINI(clean_api_env)
> +{
> + if (opae_log_file) {
> + fclose(opae_log_file);
> + opae_log_file = NULL;
> + }
> + opae_log_info("API environment is cleaned\n");
> +}
> +
> +void opae_get_api_version(opae_api_version *version)
> +{
> + if (version)
> + memcpy(version, &api_ver, sizeof(opae_api_version));
> + opae_log_info("API version is %u
[dpdk-dev] Recall: [PATCH v9 1/4] raw/ifpga: add fpga rsu function
Huang, Wei would like to recall the message, "[PATCH v9 1/4] raw/ifpga: add fpga rsu function".
Re: [dpdk-dev] [PATCH v9 1/4] raw/ifpga: add fpga rsu function
Hi,
-Original Message-
From: Xu, Rosen
Sent: Tuesday, January 12, 2021 18:46
To: Huang, Wei ; dev@dpdk.org; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei
Subject: RE: [PATCH v9 1/4] raw/ifpga: add fpga rsu function
Hi,
> -Original Message-
> From: Huang, Wei
> Sent: Tuesday, January 12, 2021 13:18
> To: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Huang,
> Wei
> Subject: [PATCH v9 1/4] raw/ifpga: add fpga rsu function
>
> RSU (Remote System Update) depends on secure manager which may be
> different on various implementations, so a new secure manager device
> is implemented for adapting such difference.
> There are three major functions added:
> 1. ifpga_rawdev_update_flash() updates flash with specific image file.
> 2. ifpga_rawdev_stop_flash_update() aborts flash update process.
> 3. ifpga_rawdev_reload() reloads FPGA from updated flash.
>
> Signed-off-by: Wei Huang
> ---
> v2: fix coding style issue in ifpga_fme_rsu.c and ifpga_sec_mgr.c
> ---
> v3: fix compilation issues in ifpga_fme_rsu.c
> ---
> v4: fix compilation issues in opae_intel_max10.c
> ---
> drivers/raw/ifpga/base/ifpga_api.c| 26 +
> drivers/raw/ifpga/base/ifpga_fme.c| 8 +
> drivers/raw/ifpga/base/ifpga_fme_rsu.c| 435 +++
> drivers/raw/ifpga/base/ifpga_hw.h | 1 +
> drivers/raw/ifpga/base/ifpga_sec_mgr.c| 639 ++
> drivers/raw/ifpga/base/ifpga_sec_mgr.h| 89 +++
> drivers/raw/ifpga/base/meson.build| 2 +
> drivers/raw/ifpga/base/opae_hw_api.c | 59 ++
> drivers/raw/ifpga/base/opae_hw_api.h | 11 +
> drivers/raw/ifpga/base/opae_intel_max10.c | 48 ++
> drivers/raw/ifpga/base/opae_intel_max10.h | 44 ++
> drivers/raw/ifpga/ifpga_rawdev.c | 55 ++
> drivers/raw/ifpga/ifpga_rawdev.h | 7 +-
> 13 files changed, 1423 insertions(+), 1 deletion(-) create mode
> 100644 drivers/raw/ifpga/base/ifpga_fme_rsu.c
> create mode 100644 drivers/raw/ifpga/base/ifpga_sec_mgr.c
> create mode 100644 drivers/raw/ifpga/base/ifpga_sec_mgr.h
>
> diff --git a/drivers/raw/ifpga/base/ifpga_api.c
> b/drivers/raw/ifpga/base/ifpga_api.c
> index 1ff57fa18..1aedf150b 100644
> --- a/drivers/raw/ifpga/base/ifpga_api.c
> +++ b/drivers/raw/ifpga/base/ifpga_api.c
> @@ -5,6 +5,7 @@
> #include "ifpga_api.h"
> #include "ifpga_enumerate.h"
> #include "ifpga_feature_dev.h"
> +#include "ifpga_sec_mgr.h"
>
> #include "opae_hw_api.h"
>
> @@ -228,11 +229,36 @@ static int ifpga_mgr_get_board_info(struct
> opae_manager *mgr,
> return 0;
> }
>
> +static int ifpga_mgr_update_flash(struct opae_manager *mgr, const
> +char
> *image,
> + u64 *status)
> +{
> + struct ifpga_fme_hw *fme = mgr->data;
> +
> + return fpga_update_flash(fme, image, status); }
> +
> +static int ifpga_mgr_stop_flash_update(struct opae_manager *mgr, int
> +force) {
> + struct ifpga_fme_hw *fme = mgr->data;
> +
> + return fpga_stop_flash_update(fme, force); }
> +
> +static int ifpga_mgr_reload(struct opae_manager *mgr, int type, int
> +page) {
> + struct ifpga_fme_hw *fme = mgr->data;
> +
> + return fpga_reload(fme, type, page); }
> +
> struct opae_manager_ops ifpga_mgr_ops = {
> .flash = ifpga_mgr_flash,
> .get_eth_group_region_info = ifpga_mgr_get_eth_group_region_info,
> .get_sensor_value = ifpga_mgr_get_sensor_value,
> .get_board_info = ifpga_mgr_get_board_info,
> + .update_flash = ifpga_mgr_update_flash,
> + .stop_flash_update = ifpga_mgr_stop_flash_update,
> + .reload = ifpga_mgr_reload,
> };
>
> static int ifpga_mgr_read_mac_rom(struct opae_manager *mgr, int
> offset, diff --git a/drivers/raw/ifpga/base/ifpga_fme.c
> b/drivers/raw/ifpga/base/ifpga_fme.c
> index f29ff3159..34fd9a818 100644
> --- a/drivers/raw/ifpga/base/ifpga_fme.c
> +++ b/drivers/raw/ifpga/base/ifpga_fme.c
> @@ -7,6 +7,7 @@
> #include "opae_intel_max10.h"
> #include "opae_i2c.h"
> #include "opae_at24_eeprom.h"
> +#include "ifpga_sec_mgr.h"
>
> #define PWR_THRESHOLD_MAX 0x7F
>
> @@ -1152,6 +1153,12 @@ static int fme_nios_spi_init(struct
> ifpga_feature
> *feature)
> if (spi_self_checking(max10))
> goto spi_fail;
>
> + ret = init_sec_mgr(fme);
> + if (ret) {
> + dev_err(fme, "security manager init fail\n");
> + goto spi_fail;
> + }
> +
> return ret;
>
> spi_fail:
> @@ -1165,6 +1172,7 @@ static void fme_nios_spi_uinit(struct
> i
[dpdk-dev] Recall: [PATCH v9 3/4] raw/ifpga: add opae API for Cyborg
Huang, Wei would like to recall the message, "[PATCH v9 3/4] raw/ifpga: add opae API for Cyborg".
Re: [dpdk-dev] [PATCH v9 3/4] raw/ifpga: add opae API for Cyborg
Hi,
-Original Message-
From: Xu, Rosen
Sent: Tuesday, January 12, 2021 19:00
To: Huang, Wei ; dev@dpdk.org; Zhang, Qi Z
Cc: sta...@dpdk.org; Zhang, Tianfei
Subject: RE: [PATCH v9 3/4] raw/ifpga: add opae API for Cyborg
Hi,
> -Original Message-
> From: Huang, Wei
> Sent: Tuesday, January 12, 2021 13:18
> To: dev@dpdk.org; Xu, Rosen ; Zhang, Qi Z
>
> Cc: sta...@dpdk.org; Zhang, Tianfei ; Huang, Wei
>
> Subject: [PATCH v9 3/4] raw/ifpga: add opae API for Cyborg
>
> Cyborg is part of OpenStack, it needs some OPAE type APIs to manage
> PACs (Programmable Acceleration Card) with Intel FPGA. Below major
> functions are added to meets Cyborg requirements.
> 1. opae_init_eal() set up EAL environment.
> 2. opae_cleanup_eal() clean up EAL environment.
> 3. opae_enumerate() searches PAC with specific FPGA.
> 4. opae_get_property() gets properties of FPGA.
> 5. opae_partial_reconfigure() perform partial configuration on FPGA.
> 6. opae_get_image_info() gets information of image file.
> 7. opae_update_flash() updates FPGA flash with specific image file.
> 8. opae_cancel_flash_update() cancel process of FPGA flash update.
> 9. opae_probe_device() manually probe specific FPGA with ifpga driver.
> 10. opae_remove_device() manually remove specific FPGA from ifpga driver.
> 11. opae_bind_driver() binds specific FPGA with specified kernel driver.
> 12. opae_unbind_driver() unbinds specific FPGA from kernel driver.
> 13. opae_reboot_device() reboots specific FPGA (do reconfiguration).
>
> Signed-off-by: Wei Huang
> ---
> v2: fix typo in commit log and ifpga_opae_api.h
> ---
> v3: fix coding style issue in ifpga_opae_api.c
> ---
> v4: enclose macro PCI_EXT_CAP_ID in parentheses
> ---
> v5: fix icc compiler build error
> ---
> v6: fix compilation issues in ifpga_opae_api.c
> ---
> drivers/raw/ifpga/ifpga_opae_api.c | 1800
>
> drivers/raw/ifpga/ifpga_opae_api.h | 245
> drivers/raw/ifpga/ifpga_rawdev.c | 152 ++-
> drivers/raw/ifpga/ifpga_rawdev.h | 15 +
> drivers/raw/ifpga/meson.build |4 +-
> 5 files changed, 2209 insertions(+), 7 deletions(-)
> create mode 100644 drivers/raw/ifpga/ifpga_opae_api.c
> create mode 100644 drivers/raw/ifpga/ifpga_opae_api.h
>
> diff --git a/drivers/raw/ifpga/ifpga_opae_api.c
> b/drivers/raw/ifpga/ifpga_opae_api.c
> new file mode 100644
> index 0..42194058c
> --- /dev/null
> +++ b/drivers/raw/ifpga/ifpga_opae_api.c
> @@ -0,0 +1,1800 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2020 Intel Corporation
> + */
> +
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include
> +#include "base/opae_hw_api.h"
> +#include "base/ifpga_sec_mgr.h"
> +#include "ifpga_rawdev.h"
> +#include "ifpga_opae_api.h"
> +
> +
> +int opae_log_level;
> +FILE *opae_log_file;
> +
> +static opae_api_version api_ver = {21, 2, 0};
> +static int eal_inited;
> +static uint32_t dev_aer[2] = {0};
> +
> +static const char * const log_level_name[] = {"CRITICAL", "ERROR",
> + "WARNING", "INFORMATION", "DEBUG"};
> +static const char * const proc_type_name[] = {"NON-DPDK", "PRIMARY",
> + "SECONDARY"};
> +static const char * const platform_name[] = {"Vista Creek", "Rush Creek",
> + "Darby Creek", "Lightning Creek"};
> +static const char * const release_name[] = {"Pre-Alpha", "Alpha", "Beta",
> "PV"};
> +static const char * const interface_type[] = {"8x10G", "4x25G", "2x1x25G",
> + "4x25G+2x25G", "2x2x25G", "2x1x25Gx2FVL", "1x2x25G"};
> +static const char * const kdrv[] = {OPAE_KDRV_UNKNOWN,
> OPAE_KDRV_IGB_UIO,
> + OPAE_KDRV_VFIO_PCI, OPAE_KDRV_UIO_PCI};
> +
> +RTE_INIT(init_api_env)
> +{
> + eal_inited = 0;
> + opae_log_level = OPAE_LOG_ERR;
> + opae_log_file = NULL;
> + ifpga_rawdev_logtype = 0;
> +
> + opae_log_info("API environment is initialized\n");
> +}
> +
> +RTE_FINI(clean_api_env)
> +{
> + if (opae_log_file) {
> + fclose(opae_log_file);
> + opae_log_file = NULL;
> + }
> + opae_log_info("API environment is cleaned\n");
> +}
> +
> +void opae_get_api_version(opae_api_version *version)
> +{
> + if (version)
> + memcpy(version, &api_ver, sizeof(opae_api_version));
> + opae_log_info("API version is %u
