Re: [PATCH v2 2/2] random: make rte_rand() thread safe for non-EAL threads
On 2023-09-08 22:56, Stephen Hemminger wrote:
On Fri, 8 Sep 2023 22:48:54 +0200
Mattias Rönnblom wrote:
On 2023-09-07 17:24, Stephen Hemminger wrote:
Add missing locking so that if two non-EAL threads call rte_rand()
they will not corrupt the per-thread state.
Fixes: 3f002f069612 ("eal: replace libc-based random generation with LFSR")
The API documentation clearly states that no MT safety guarantees are
given for unregistered non-EAL threads. So this patch doesn't fix anything.
rte_rand() is MT safe for *registered* non-EAL threads.
Reading the documentation, it only guarantees safety if registered.
We should add an otherwise clause rather than leaving case as undefined.
I agree. It is MT safe in case only single unregistered non-EAL thread
uses the API (or multiple such threads, provided they are properly
[externally] serialized).
"If called from EAL threads, registered non-EAL threads or a *single*
unregistered non-EAL thread, this function is thread-safe. Multiple
unregistered non-EAL threads may not safely call this function in
parallel (i.e., must use external serialization)."
A lot of words, unfortunately. Maybe this is better:
"rte_xxx() is MT safe, with the exception it may not be called from
multiple *unregistered* non-EAL threads in parallel."
Then of course comes the issue that nobody knows what is an registered
non-EAL thread is. DPDK threading terminology is a big mess, so no wonder.
RE: [RFC] random: use per lcore state
> From: Mattias Rönnblom [mailto:hof...@lysator.liu.se]
> Sent: Saturday, 9 September 2023 08.45
>
> On 2023-09-09 02:13, Konstantin Ananyev wrote:
> > 06/09/2023 21:02, Mattias Rönnblom пишет:
> >> On 2023-09-06 19:20, Stephen Hemminger wrote:
> >>> Move the random number state into thread local storage.
> >>
> >> Me and Morten discussed TLS versus other alternatives in some other
> >> thread. The downside of TLS that Morten pointed out, from what I
> >> recall, is that lazy initialization is *required* (since the number
> of
> >> threads is open-ended), and the data ends up in non-huge page memory.
> >
> > Hmm.. correct me if I am wrong, but with current implementation,
> > rand state is also in non-huge memory:
> > static struct rte_rand_state rand_states[RTE_MAX_LCORE + 1];
> >
>
> Yes. The current pattern is certainly not perfect.
>
> >
> >> It was also unclear to me what the memory footprint implications
> would
> >> be,h would large per-lcore data structures be put in TLS. More
> >> specifically, if they would be duplicated across all threads, even
> >> non-lcore threads.
> >>
> >> None of these issues affect rte_random.c's potential usage of TLS
> >> (except lazy [re-]initialization makes things more complicated).
> >>
> >> Preferably, there should be one pattern that is usable across all or
> >> at least most DPDK modules requiring per-lcore state.
> >>
> >>> This has a several benefits.
> >>> - no false cache sharing from cpu prefetching
> >>> - fixes initialization of random state for non-DPDK threads
> >>
> >> This seems like a non-reason to me. That bug is easily fixed, if it
> >> isn't already.
> >>
> >>> - fixes unsafe usage of random state by non-DPDK threads
> >>>
> >>
> >> "Makes random number generation MT safe from all threads (including
> >> unregistered non-EAL threads)."
> >>
> >> With current API semantics you may still register an non-EAL thread,
> >> to get MT safe access to this API, so I guess it's more about being
> >> more convenient and less error prone, than anything else.
> >
> > I understand that we never guaranteed MT safety for non-EAL threads
> here,
>
>
> Registered non-EAL threads have a lcore id and thus may safely call
> rte_rand(). Multiple unregistered non-EAL threads may not do so, in
> parallel.
>
>
> > but as a user of rte_rand() - it would be much more convenient, if I
> can
> > use it
> > from any thread wthout worring is it a EAL thread or not.
>
> Sure, especially if it comes for free. The for-free solution has yet to
> reveal itself though.
We could offer re-entrant function variants for non-EAL threads:
uint64_t rte_rand_r(struct rte_rand_state * const state);
void rte_srand_r(struct rte_rand_state * const state, uint64_t seed);
uint64_t rte_rand_max_r(struct rte_rand_state * const state, uint64_t
upper_bound);
double rte_drand_r(struct rte_rand_state * const state, void);
For this to work, we would have to make struct rte_rand_state public, and the
application would need to allocate it. (At least one instance per thread that
uses it, obviously.)
>
> >
> > About TlS usage and re-seeding - can we use some sort of middle-
> ground:
> > extend rte_rand_state with some gen-counter.
> > Make a 'master' copy of rte_rand_state that will be updated by
> rte_srand(),
> > and TLS copies of rte_rand_state, so rte_rand() can fist compare
> > its gen-counter value with master copy to decide,
> > does it need to copy new state from master or not.
> >
>
> Calling threads shouldn't all produce the same sequence. That would be
> silly and not very random. The generation number should be tied to the
> seed.
I previously thought about seeding...
We are trying to be random, we are not explicitly pseudo-random.
So I came to the conclusion that the ability to reproduce data (typically for
verification purposes) is not a requirement here.
>
> >
> >> The new MT safety guarantees should be in the API docs as well.
> >
> > Yes, it is an extension to the current API, not a fix.
> >
> >>
> >>> The initialization of random number state is done by the
> >>> lcore (lazy initialization).
> >>>
> >>> Signed-off-by: Stephen Hemminger
> >>> ---
> >>> lib/eal/common/rte_random.c | 38 +++--
>
> >>> 1 file changed, 20 insertions(+), 18 deletions(-)
> >>>
> >>> diff --git a/lib/eal/common/rte_random.c
> b/lib/eal/common/rte_random.c
> >>> index 53636331a27b..9657adf6ad3b 100644
> >>> --- a/lib/eal/common/rte_random.c
> >>> +++ b/lib/eal/common/rte_random.c
> >>> @@ -19,13 +19,14 @@ struct rte_rand_state {
> >>> uint64_t z3;
> >>> uint64_t z4;
> >>> uint64_t z5;
> >>> -} __rte_cache_aligned;
> >>> + uint64_t seed;
> >>> +};
> >>> -/* One instance each for every lcore id-equipped thread, and one
> >>> - * additional instance to be shared by all others threads (i.e.,
> all
> >>> - * unregistered non-EAL threads).
> >>> - */
> >>> -static struct rte_rand_state rand_states[RTE_MAX_LCORE + 1];
> >>> +/* Global ra
[PATCH 1/1] net/mana: enable 32 bit build for mana driver
Enable 32 bit build on x86 Linux. Fixed build warnings and errors
when building in 32 bit.
Cc: sta...@dpdk.org
Signed-off-by: Wei Hu
---
drivers/net/mana/mana.c | 2 +-
drivers/net/mana/meson.build | 4 ++--
drivers/net/mana/mr.c| 18 +-
3 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/net/mana/mana.c b/drivers/net/mana/mana.c
index 7630118d4f..a20ca1a988 100644
--- a/drivers/net/mana/mana.c
+++ b/drivers/net/mana/mana.c
@@ -1260,7 +1260,7 @@ mana_probe_port(struct ibv_device *ibdev, struct
ibv_device_attr_ex *dev_attr,
/* Create a parent domain with the port number */
attr.pd = priv->ib_pd;
attr.comp_mask = IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT;
- attr.pd_context = (void *)(uint64_t)port;
+ attr.pd_context = (void *)(size_t)port;
priv->ib_parent_pd = ibv_alloc_parent_domain(ctx, &attr);
if (!priv->ib_parent_pd) {
DRV_LOG(ERR, "ibv_alloc_parent_domain failed port %d", port);
diff --git a/drivers/net/mana/meson.build b/drivers/net/mana/meson.build
index 493f0d26d4..2d72eca5a8 100644
--- a/drivers/net/mana/meson.build
+++ b/drivers/net/mana/meson.build
@@ -1,9 +1,9 @@
# SPDX-License-Identifier: BSD-3-Clause
# Copyright(c) 2022 Microsoft Corporation
-if not is_linux or not dpdk_conf.has('RTE_ARCH_X86_64')
+if not is_linux or not dpdk_conf.has('RTE_ARCH_X86')
build = false
-reason = 'only supported on x86_64 Linux'
+reason = 'only supported on x86 Linux'
subdir_done()
endif
diff --git a/drivers/net/mana/mr.c b/drivers/net/mana/mr.c
index fec0dc961c..b8e6ea0bbf 100644
--- a/drivers/net/mana/mr.c
+++ b/drivers/net/mana/mr.c
@@ -53,7 +53,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct
mana_priv *priv,
}
DP_LOG(DEBUG,
- "registering memory chunk start 0x%" PRIx64 " len %u",
+ "registering memory chunk start 0x%" PRIxPTR " len %u",
ranges[i].start, ranges[i].len);
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
@@ -62,7 +62,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct
mana_priv *priv,
ranges[i].len);
if (ret) {
DP_LOG(ERR,
- "MR failed start 0x%" PRIx64 " len %u",
+ "MR failed start 0x%" PRIxPTR " len %u",
ranges[i].start, ranges[i].len);
return ret;
}
@@ -72,7 +72,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct
mana_priv *priv,
ibv_mr = ibv_reg_mr(priv->ib_pd, (void *)ranges[i].start,
ranges[i].len, IBV_ACCESS_LOCAL_WRITE);
if (ibv_mr) {
- DP_LOG(DEBUG, "MR lkey %u addr %p len %" PRIu64,
+ DP_LOG(DEBUG, "MR lkey %u addr %p len %zu",
ibv_mr->lkey, ibv_mr->addr, ibv_mr->length);
mr = rte_calloc("MANA MR", 1, sizeof(*mr), 0);
@@ -99,7 +99,7 @@ mana_new_pmd_mr(struct mana_mr_btree *local_tree, struct
mana_priv *priv,
return ret;
}
} else {
- DP_LOG(ERR, "MR failed at 0x%" PRIx64 " len %u",
+ DP_LOG(ERR, "MR failed at 0x%" PRIxPTR " len %u",
ranges[i].start, ranges[i].len);
return -errno;
}
@@ -141,7 +141,7 @@ mana_find_pmd_mr(struct mana_mr_btree *local_mr_btree,
struct mana_priv *priv,
mr = mana_mr_btree_lookup(local_mr_btree, &idx,
(uintptr_t)mbuf->buf_addr, mbuf->buf_len);
if (mr) {
- DP_LOG(DEBUG, "Local mr lkey %u addr 0x%" PRIx64 " len %"
PRIu64,
+ DP_LOG(DEBUG, "Local mr lkey %u addr 0x%" PRIxPTR " len %zu",
mr->lkey, mr->addr, mr->len);
return mr;
}
@@ -162,7 +162,7 @@ mana_find_pmd_mr(struct mana_mr_btree *local_mr_btree,
struct mana_priv *priv,
}
DP_LOG(DEBUG,
- "Added local MR key %u addr 0x%" PRIx64 " len %" PRIu64,
+ "Added local MR key %u addr 0x%" PRIxPTR " len %zu",
mr->lkey, mr->addr, mr->len);
return mr;
}
@@ -266,7 +266,7 @@ mana_mr_btree_lookup(struct mana_mr_btree *bt, uint16_t
*idx,
return &table[base];
DP_LOG(DEBUG,
- "addr 0x%" PRIx64 " len %zu idx %u sum 0x%" PRIx64 " not found",
+ "addr 0x%" PRIxPTR " len %zu idx %u sum 0x%" PRIxPTR " not
found",
addr, len, *idx, addr + len);
return NULL;
@@ -316,7 +316,7 @@ mana_mr_btree_insert(s
[PATCH 1/1] net/mana: add 32 bit short doorbell
Add 32 bit short doorbell support. Ring short doorbell when running
in 32 bit applicactions.
Cc: sta...@dpdk.org
Signed-off-by: Wei Hu
---
drivers/net/mana/gdma.c | 95 +
drivers/net/mana/mana.h | 25 +++
drivers/net/mana/rx.c | 52 ++
drivers/net/mana/tx.c | 28
4 files changed, 200 insertions(+)
diff --git a/drivers/net/mana/gdma.c b/drivers/net/mana/gdma.c
index 65685fe236..d1da025d1b 100644
--- a/drivers/net/mana/gdma.c
+++ b/drivers/net/mana/gdma.c
@@ -166,6 +166,97 @@ gdma_post_work_request(struct mana_gdma_queue *queue,
return 0;
}
+#ifdef RTE_ARCH_32
+union gdma_short_doorbell_entry {
+ uint32_t as_uint32;
+
+ struct {
+ uint32_t tail_ptr_incr : 16; /* Number of CQEs */
+ uint32_t id : 12;
+ uint32_t reserved : 3;
+ uint32_t arm: 1;
+ } cq;
+
+ struct {
+ uint32_t tail_ptr_incr : 16; /* In number of bytes */
+ uint32_t id : 12;
+ uint32_t reserved : 4;
+ } rq;
+
+ struct {
+ uint32_t tail_ptr_incr : 16; /* In number of bytes */
+ uint32_t id : 12;
+ uint32_t reserved : 4;
+ } sq;
+
+ struct {
+ uint32_t tail_ptr_incr : 16; /* Number of EQEs */
+ uint32_t id : 12;
+ uint32_t reserved : 3;
+ uint32_t arm: 1;
+ } eq;
+}; /* HW DATA */
+
+enum {
+ DOORBELL_SHORT_OFFSET_SQ = 0x10,
+ DOORBELL_SHORT_OFFSET_RQ = 0x410,
+ DOORBELL_SHORT_OFFSET_CQ = 0x810,
+ DOORBELL_SHORT_OFFSET_EQ = 0xFF0,
+};
+
+/*
+ * Write to hardware doorbell to notify new activity.
+ */
+int
+mana_ring_short_doorbell(void *db_page, enum gdma_queue_types queue_type,
+uint32_t queue_id, uint32_t tail_incr, uint8_t arm)
+{
+ uint8_t *addr = db_page;
+ union gdma_short_doorbell_entry e = {};
+
+ if ((queue_id & ~GDMA_SHORT_DB_QID_MASK) ||
+ (tail_incr & ~GDMA_SHORT_DB_INC_MASK)) {
+ DP_LOG(ERR, "%s: queue_id %u or "
+ "tail_incr %u overflowed, queue type %d",
+ __func__, queue_id, tail_incr, queue_type);
+ return -EINVAL;
+ }
+
+ switch (queue_type) {
+ case GDMA_QUEUE_SEND:
+ e.sq.id = queue_id;
+ e.sq.tail_ptr_incr = tail_incr;
+ addr += DOORBELL_SHORT_OFFSET_SQ;
+ break;
+
+ case GDMA_QUEUE_RECEIVE:
+ e.rq.id = queue_id;
+ e.rq.tail_ptr_incr = tail_incr;
+ addr += DOORBELL_SHORT_OFFSET_RQ;
+ break;
+
+ case GDMA_QUEUE_COMPLETION:
+ e.cq.id = queue_id;
+ e.cq.tail_ptr_incr = tail_incr;
+ e.cq.arm = arm;
+ addr += DOORBELL_SHORT_OFFSET_CQ;
+ break;
+
+ default:
+ DP_LOG(ERR, "Unsupported queue type %d", queue_type);
+ return -1;
+ }
+
+ /* Ensure all writes are done before ringing doorbell */
+ rte_wmb();
+
+ DP_LOG(DEBUG, "db_page %p addr %p queue_id %u type %u tail %u arm %u",
+ db_page, addr, queue_id, queue_type, tail_incr, arm);
+
+ rte_write32(e.as_uint32, addr);
+ return 0;
+}
+#else
union gdma_doorbell_entry {
uint64_t as_uint64;
@@ -248,6 +339,7 @@ mana_ring_doorbell(void *db_page, enum gdma_queue_types
queue_type,
rte_write64(e.as_uint64, addr);
return 0;
}
+#endif
/*
* Poll completion queue for completions.
@@ -287,6 +379,9 @@ gdma_poll_completion_queue(struct mana_gdma_queue *cq,
num_comp++;
cq->head++;
+#ifdef RTE_ARCH_32
+ cq->head_incr_to_short_db++;
+#endif
DP_LOG(DEBUG, "comp new 0x%x old 0x%x cqe 0x%x wq %u sq %u head
%u",
new_owner_bits, old_owner_bits, cqe_owner_bits,
diff --git a/drivers/net/mana/mana.h b/drivers/net/mana/mana.h
index 5801491d75..848d87c096 100644
--- a/drivers/net/mana/mana.h
+++ b/drivers/net/mana/mana.h
@@ -50,6 +50,19 @@ struct mana_shared_data {
#define MAX_TX_WQE_SIZE 512
#define MAX_RX_WQE_SIZE 256
+/* For 32 bit only */
+#ifdef RTE_ARCH_32
+#defineGDMA_SHORT_DB_INC_MASK 0x
+#defineGDMA_SHORT_DB_QID_MASK 0xfff
+
+#define GDMA_SHORT_DB_MAX_WQE (0x1 / GDMA_WQE_ALIGNMENT_UNIT_SIZE)
+
+#define TX_WQE_SHORT_DB_THRESHOLD \
+ (GDMA_SHORT_DB_MAX_WQE - (2 * MAX_TX_WQE_SIZE))
+#define RX_WQE_SHORT_DB_THRESHOLD \
+ (GDMA_SHORT_DB_MAX_WQE - (2 * MAX_RX_WQE_SIZE))
+#endif
+
/* Values from the GDMA specification document, WQE format description */
#define INLINE_OOB_SMALL_SIZE_IN_BYTES 8
#define INLINE_OOB_LARGE_SIZE_IN_BYTES 24
@@ -375,6 +388,9 @@ struct mana
Re: [PATCH 1/1] net/mana: enable 32 bit build for mana driver
On Sat, 9 Sep 2023 12:18:30 + Wei Hu wrote: > diff --git a/drivers/net/mana/mana.c b/drivers/net/mana/mana.c > index 7630118d4f..a20ca1a988 100644 > --- a/drivers/net/mana/mana.c > +++ b/drivers/net/mana/mana.c > @@ -1260,7 +1260,7 @@ mana_probe_port(struct ibv_device *ibdev, struct > ibv_device_attr_ex *dev_attr, > /* Create a parent domain with the port number */ > attr.pd = priv->ib_pd; > attr.comp_mask = IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT; > - attr.pd_context = (void *)(uint64_t)port; > + attr.pd_context = (void *)(size_t)port; Since port is uint8_t, the better cast would be to uintptr_t which is always an unsigned value of same size as pointer. Also, not sure why using PRIxPTR is necessary; on all arch and platforms %p should work for printing a pointer and is more common usage.
[PATCH v6 1/2] dma/cnxk: rework DMA driver
From: Pavan Nikhilesh
To use the mempool cache, use rte_mempool for the DMA chunk pool.
Move the mempool creation to device start to limit the number of
chunks allocated based on the total number of descriptors
configured across all the vchans.
Remove unnecessary state tracking flags as the library handles it
and add the `CNXK` prefix to driver macros.
Convert the log register macro for all cnxk drivers to
RTE_LOG_REGISTER_DEFAULT.
Signed-off-by: Pavan Nikhilesh
---
Depends-on: 29324
v6 Changes:
- Rework drvice configuration and start logic.
- add CNXK prefix to driver macros.
v5 Changes:
- Use RTE_LOG_REGISTER_DEFAULT for registering logging.
v4 Changes:
- Fix clang build.
v3 Changes:
- Fix build.
drivers/common/cnxk/roc_dpi.c | 90 ++-
drivers/common/cnxk/roc_dpi.h | 28 +---
drivers/common/cnxk/roc_dpi_priv.h | 3 -
drivers/common/cnxk/roc_platform.c | 21 +--
drivers/common/cnxk/roc_platform.h | 2 +
drivers/common/cnxk/version.map| 1 +
drivers/dma/cnxk/cnxk_dmadev.c | 252 -
drivers/dma/cnxk/cnxk_dmadev.h | 45 --
8 files changed, 203 insertions(+), 239 deletions(-)
diff --git a/drivers/common/cnxk/roc_dpi.c b/drivers/common/cnxk/roc_dpi.c
index 0e2f803077..c241168294 100644
--- a/drivers/common/cnxk/roc_dpi.c
+++ b/drivers/common/cnxk/roc_dpi.c
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright(C) 2021 Marvell.
*/
+
#include
#include
#include
@@ -52,17 +53,12 @@ roc_dpi_disable(struct roc_dpi *dpi)
}
int
-roc_dpi_configure(struct roc_dpi *roc_dpi)
+roc_dpi_configure(struct roc_dpi *roc_dpi, uint32_t chunk_sz, uint64_t aura,
uint64_t chunk_base)
{
struct plt_pci_device *pci_dev;
- const struct plt_memzone *dpi_mz;
dpi_mbox_msg_t mbox_msg;
- struct npa_pool_s pool;
- struct npa_aura_s aura;
- int rc, count, buflen;
- uint64_t aura_handle;
- plt_iova_t iova;
- char name[32];
+ uint64_t reg;
+ int rc;
if (!roc_dpi) {
plt_err("roc_dpi is NULL");
@@ -70,79 +66,30 @@ roc_dpi_configure(struct roc_dpi *roc_dpi)
}
pci_dev = roc_dpi->pci_dev;
- memset(&pool, 0, sizeof(struct npa_pool_s));
- pool.nat_align = 1;
-
- memset(&aura, 0, sizeof(aura));
- rc = roc_npa_pool_create(&aura_handle, DPI_CMD_QUEUE_SIZE,
-DPI_CMD_QUEUE_BUFS, &aura, &pool, 0);
- if (rc) {
- plt_err("Failed to create NPA pool, err %d\n", rc);
- return rc;
- }
-
- snprintf(name, sizeof(name), "dpimem%d:%d:%d:%d", pci_dev->addr.domain,
pci_dev->addr.bus,
-pci_dev->addr.devid, pci_dev->addr.function);
- buflen = DPI_CMD_QUEUE_SIZE * DPI_CMD_QUEUE_BUFS;
- dpi_mz = plt_memzone_reserve_aligned(name, buflen, 0,
DPI_CMD_QUEUE_SIZE);
- if (dpi_mz == NULL) {
- plt_err("dpi memzone reserve failed");
- rc = -ENOMEM;
- goto err1;
- }
-
- roc_dpi->mz = dpi_mz;
- iova = dpi_mz->iova;
- for (count = 0; count < DPI_CMD_QUEUE_BUFS; count++) {
- roc_npa_aura_op_free(aura_handle, 0, iova);
- iova += DPI_CMD_QUEUE_SIZE;
- }
-
- roc_dpi->chunk_base = (void *)roc_npa_aura_op_alloc(aura_handle, 0);
- if (!roc_dpi->chunk_base) {
- plt_err("Failed to alloc buffer from NPA aura");
- rc = -ENOMEM;
- goto err2;
- }
-
- roc_dpi->chunk_next = (void *)roc_npa_aura_op_alloc(aura_handle, 0);
- if (!roc_dpi->chunk_next) {
- plt_err("Failed to alloc buffer from NPA aura");
- rc = -ENOMEM;
- goto err2;
- }
- roc_dpi->aura_handle = aura_handle;
- /* subtract 2 as they have already been alloc'ed above */
- roc_dpi->pool_size_m1 = (DPI_CMD_QUEUE_SIZE >> 3) - 2;
+ roc_dpi_disable(roc_dpi);
+ reg = plt_read64(roc_dpi->rbase + DPI_VDMA_SADDR);
+ while (!(reg & BIT_ULL(63)))
+ reg = plt_read64(roc_dpi->rbase + DPI_VDMA_SADDR);
plt_write64(0x0, roc_dpi->rbase + DPI_VDMA_REQQ_CTL);
- plt_write64(((uint64_t)(roc_dpi->chunk_base) >> 7) << 7,
- roc_dpi->rbase + DPI_VDMA_SADDR);
+ plt_write64(chunk_base, roc_dpi->rbase + DPI_VDMA_SADDR);
mbox_msg.u[0] = 0;
mbox_msg.u[1] = 0;
/* DPI PF driver expects vfid starts from index 0 */
mbox_msg.s.vfid = roc_dpi->vfid;
mbox_msg.s.cmd = DPI_QUEUE_OPEN;
- mbox_msg.s.csize = DPI_CMD_QUEUE_SIZE;
- mbox_msg.s.aura = roc_npa_aura_handle_to_aura(aura_handle);
+ mbox_msg.s.csize = chunk_sz;
+ mbox_msg.s.aura = aura;
mbox_msg.s.sso_pf_func = idev_sso_pffunc_get();
mbox_msg.s.npa_pf_func = idev_npa_pffunc_get();
rc = send_msg_to_pf(&pci_dev->addr, (const char *)&mbox_msg,
size
[PATCH v6 2/2] dma/cnxk: rewrite DMA fastpath
From: Pavan Nikhilesh
Rewrite DMA fastpath to use NEON instructions and reduce number
of words read from config.
Signed-off-by: Pavan Nikhilesh
---
drivers/dma/cnxk/cnxk_dmadev.c| 428 ++---
drivers/dma/cnxk/cnxk_dmadev.h| 59 +++-
drivers/dma/cnxk/cnxk_dmadev_fp.c | 436 ++
drivers/dma/cnxk/meson.build | 9 +-
4 files changed, 528 insertions(+), 404 deletions(-)
create mode 100644 drivers/dma/cnxk/cnxk_dmadev_fp.c
diff --git a/drivers/dma/cnxk/cnxk_dmadev.c b/drivers/dma/cnxk/cnxk_dmadev.c
index f58bb92dbc..26680edfde 100644
--- a/drivers/dma/cnxk/cnxk_dmadev.c
+++ b/drivers/dma/cnxk/cnxk_dmadev.c
@@ -115,19 +115,9 @@ cnxk_dmadev_configure(struct rte_dma_dev *dev, const
struct rte_dma_conf *conf,
return 0;
}
-static int
-cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
- const struct rte_dma_vchan_conf *conf, uint32_t conf_sz)
+static void
+cn9k_dmadev_setup_hdr(union cnxk_dpi_instr_cmd *header, const struct
rte_dma_vchan_conf *conf)
{
- struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
- struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
- union dpi_instr_hdr_s *header = &dpi_conf->hdr;
- uint16_t max_desc;
- uint32_t size;
- int i;
-
- RTE_SET_USED(conf_sz);
-
header->cn9k.pt = DPI_HDR_PT_ZBW_CA;
switch (conf->direction) {
@@ -163,54 +153,11 @@ cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t
vchan,
header->cn9k.fport = conf->dst_port.pcie.coreid;
header->cn9k.pvfe = 0;
};
-
- /* Free up descriptor memory before allocating. */
- cnxk_dmadev_vchan_free(dpivf, vchan);
-
- max_desc = conf->nb_desc;
- if (!rte_is_power_of_2(max_desc))
- max_desc = rte_align32pow2(max_desc);
-
- if (max_desc > CNXK_DPI_MAX_DESC)
- max_desc = CNXK_DPI_MAX_DESC;
-
- size = (max_desc * sizeof(struct cnxk_dpi_compl_s *));
- dpi_conf->c_desc.compl_ptr = rte_zmalloc(NULL, size, 0);
-
- if (dpi_conf->c_desc.compl_ptr == NULL) {
- plt_err("Failed to allocate for comp_data");
- return -ENOMEM;
- }
-
- for (i = 0; i < max_desc; i++) {
- dpi_conf->c_desc.compl_ptr[i] =
- rte_zmalloc(NULL, sizeof(struct cnxk_dpi_compl_s), 0);
- if (!dpi_conf->c_desc.compl_ptr[i]) {
- plt_err("Failed to allocate for descriptor memory");
- return -ENOMEM;
- }
-
- dpi_conf->c_desc.compl_ptr[i]->cdata = CNXK_DPI_REQ_CDATA;
- }
-
- dpi_conf->c_desc.max_cnt = (max_desc - 1);
-
- return 0;
}
-static int
-cn10k_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
-const struct rte_dma_vchan_conf *conf, uint32_t
conf_sz)
+static void
+cn10k_dmadev_setup_hdr(union cnxk_dpi_instr_cmd *header, const struct
rte_dma_vchan_conf *conf)
{
- struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
- struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
- union dpi_instr_hdr_s *header = &dpi_conf->hdr;
- uint16_t max_desc;
- uint32_t size;
- int i;
-
- RTE_SET_USED(conf_sz);
-
header->cn10k.pt = DPI_HDR_PT_ZBW_CA;
switch (conf->direction) {
@@ -246,6 +193,27 @@ cn10k_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t
vchan,
header->cn10k.fport = conf->dst_port.pcie.coreid;
header->cn10k.pvfe = 0;
};
+}
+
+static int
+cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
+ const struct rte_dma_vchan_conf *conf, uint32_t conf_sz)
+{
+ struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
+ struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
+ union cnxk_dpi_instr_cmd *header;
+ uint16_t max_desc;
+ uint32_t size;
+ int i;
+
+ RTE_SET_USED(conf_sz);
+
+ header = (union cnxk_dpi_instr_cmd *)&dpi_conf->cmd.u;
+
+ if (dpivf->is_cn10k)
+ cn10k_dmadev_setup_hdr(header, conf);
+ else
+ cn9k_dmadev_setup_hdr(header, conf);
/* Free up descriptor memory before allocating. */
cnxk_dmadev_vchan_free(dpivf, vchan);
@@ -371,333 +339,6 @@ cnxk_dmadev_close(struct rte_dma_dev *dev)
return 0;
}
-static inline int
-__dpi_queue_write(struct cnxk_dpi_vf_s *dpi, uint64_t *cmds, int cmd_count)
-{
- uint64_t *ptr = dpi->chunk_base;
-
- if ((cmd_count < DPI_MIN_CMD_SIZE) || (cmd_count > DPI_MAX_CMD_SIZE) ||
cmds == NULL)
- return -EINVAL;
-
- /*
-* Normally there is plenty of room in the current buffer for the
-* command
-*/
- if (dpi->chunk_head + cmd_count < dpi->chunk_size_m1) {
- ptr += dpi->chunk_head;
- dpi->chunk_head +=
[PATCH v7 1/2] dma/cnxk: rework DMA driver
From: Pavan Nikhilesh
To use the mempool cache, use rte_mempool for the DMA chunk pool.
Move the mempool creation to device start to limit the number of
chunks allocated based on the total number of descriptors
configured across all the vchans.
Remove unnecessary state tracking flags as the library handles it
and add the `CNXK` prefix to driver macros.
Convert the log register macro for all cnxk drivers to
RTE_LOG_REGISTER_DEFAULT.
Signed-off-by: Pavan Nikhilesh
---
Depends-on: 29324
v7 Changes:
- Fix checkpatch warnings.
v6 Changes:
- Rework drvice configuration and start logic.
- add CNXK prefix to driver macros.
v5 Changes:
- Use RTE_LOG_REGISTER_DEFAULT for registering logging.
v4 Changes:
- Fix clang build.
v3 Changes:
- Fix build.
drivers/common/cnxk/roc_dpi.c | 90 ++-
drivers/common/cnxk/roc_dpi.h | 28 +---
drivers/common/cnxk/roc_dpi_priv.h | 3 -
drivers/common/cnxk/roc_platform.c | 21 +--
drivers/common/cnxk/roc_platform.h | 2 +
drivers/common/cnxk/version.map| 1 +
drivers/dma/cnxk/cnxk_dmadev.c | 252 -
drivers/dma/cnxk/cnxk_dmadev.h | 45 --
8 files changed, 203 insertions(+), 239 deletions(-)
diff --git a/drivers/common/cnxk/roc_dpi.c b/drivers/common/cnxk/roc_dpi.c
index 0e2f803077..c241168294 100644
--- a/drivers/common/cnxk/roc_dpi.c
+++ b/drivers/common/cnxk/roc_dpi.c
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright(C) 2021 Marvell.
*/
+
#include
#include
#include
@@ -52,17 +53,12 @@ roc_dpi_disable(struct roc_dpi *dpi)
}
int
-roc_dpi_configure(struct roc_dpi *roc_dpi)
+roc_dpi_configure(struct roc_dpi *roc_dpi, uint32_t chunk_sz, uint64_t aura,
uint64_t chunk_base)
{
struct plt_pci_device *pci_dev;
- const struct plt_memzone *dpi_mz;
dpi_mbox_msg_t mbox_msg;
- struct npa_pool_s pool;
- struct npa_aura_s aura;
- int rc, count, buflen;
- uint64_t aura_handle;
- plt_iova_t iova;
- char name[32];
+ uint64_t reg;
+ int rc;
if (!roc_dpi) {
plt_err("roc_dpi is NULL");
@@ -70,79 +66,30 @@ roc_dpi_configure(struct roc_dpi *roc_dpi)
}
pci_dev = roc_dpi->pci_dev;
- memset(&pool, 0, sizeof(struct npa_pool_s));
- pool.nat_align = 1;
-
- memset(&aura, 0, sizeof(aura));
- rc = roc_npa_pool_create(&aura_handle, DPI_CMD_QUEUE_SIZE,
-DPI_CMD_QUEUE_BUFS, &aura, &pool, 0);
- if (rc) {
- plt_err("Failed to create NPA pool, err %d\n", rc);
- return rc;
- }
-
- snprintf(name, sizeof(name), "dpimem%d:%d:%d:%d", pci_dev->addr.domain,
pci_dev->addr.bus,
-pci_dev->addr.devid, pci_dev->addr.function);
- buflen = DPI_CMD_QUEUE_SIZE * DPI_CMD_QUEUE_BUFS;
- dpi_mz = plt_memzone_reserve_aligned(name, buflen, 0,
DPI_CMD_QUEUE_SIZE);
- if (dpi_mz == NULL) {
- plt_err("dpi memzone reserve failed");
- rc = -ENOMEM;
- goto err1;
- }
-
- roc_dpi->mz = dpi_mz;
- iova = dpi_mz->iova;
- for (count = 0; count < DPI_CMD_QUEUE_BUFS; count++) {
- roc_npa_aura_op_free(aura_handle, 0, iova);
- iova += DPI_CMD_QUEUE_SIZE;
- }
-
- roc_dpi->chunk_base = (void *)roc_npa_aura_op_alloc(aura_handle, 0);
- if (!roc_dpi->chunk_base) {
- plt_err("Failed to alloc buffer from NPA aura");
- rc = -ENOMEM;
- goto err2;
- }
-
- roc_dpi->chunk_next = (void *)roc_npa_aura_op_alloc(aura_handle, 0);
- if (!roc_dpi->chunk_next) {
- plt_err("Failed to alloc buffer from NPA aura");
- rc = -ENOMEM;
- goto err2;
- }
- roc_dpi->aura_handle = aura_handle;
- /* subtract 2 as they have already been alloc'ed above */
- roc_dpi->pool_size_m1 = (DPI_CMD_QUEUE_SIZE >> 3) - 2;
+ roc_dpi_disable(roc_dpi);
+ reg = plt_read64(roc_dpi->rbase + DPI_VDMA_SADDR);
+ while (!(reg & BIT_ULL(63)))
+ reg = plt_read64(roc_dpi->rbase + DPI_VDMA_SADDR);
plt_write64(0x0, roc_dpi->rbase + DPI_VDMA_REQQ_CTL);
- plt_write64(((uint64_t)(roc_dpi->chunk_base) >> 7) << 7,
- roc_dpi->rbase + DPI_VDMA_SADDR);
+ plt_write64(chunk_base, roc_dpi->rbase + DPI_VDMA_SADDR);
mbox_msg.u[0] = 0;
mbox_msg.u[1] = 0;
/* DPI PF driver expects vfid starts from index 0 */
mbox_msg.s.vfid = roc_dpi->vfid;
mbox_msg.s.cmd = DPI_QUEUE_OPEN;
- mbox_msg.s.csize = DPI_CMD_QUEUE_SIZE;
- mbox_msg.s.aura = roc_npa_aura_handle_to_aura(aura_handle);
+ mbox_msg.s.csize = chunk_sz;
+ mbox_msg.s.aura = aura;
mbox_msg.s.sso_pf_func = idev_sso_pffunc_get();
mbox_msg.s.npa_pf_func = idev_npa_pffunc_get();
rc = send_msg_to_pf(&pci_dev->addr, (const char *)&m
[PATCH v7 2/2] dma/cnxk: rewrite DMA fastpath
From: Pavan Nikhilesh
Rewrite DMA fastpath to use NEON instructions and reduce number
of words read from config.
Signed-off-by: Pavan Nikhilesh
---
drivers/dma/cnxk/cnxk_dmadev.c| 428 ++---
drivers/dma/cnxk/cnxk_dmadev.h| 59 +++-
drivers/dma/cnxk/cnxk_dmadev_fp.c | 436 ++
drivers/dma/cnxk/meson.build | 9 +-
4 files changed, 528 insertions(+), 404 deletions(-)
create mode 100644 drivers/dma/cnxk/cnxk_dmadev_fp.c
diff --git a/drivers/dma/cnxk/cnxk_dmadev.c b/drivers/dma/cnxk/cnxk_dmadev.c
index f58bb92dbc..26680edfde 100644
--- a/drivers/dma/cnxk/cnxk_dmadev.c
+++ b/drivers/dma/cnxk/cnxk_dmadev.c
@@ -115,19 +115,9 @@ cnxk_dmadev_configure(struct rte_dma_dev *dev, const
struct rte_dma_conf *conf,
return 0;
}
-static int
-cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
- const struct rte_dma_vchan_conf *conf, uint32_t conf_sz)
+static void
+cn9k_dmadev_setup_hdr(union cnxk_dpi_instr_cmd *header, const struct
rte_dma_vchan_conf *conf)
{
- struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
- struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
- union dpi_instr_hdr_s *header = &dpi_conf->hdr;
- uint16_t max_desc;
- uint32_t size;
- int i;
-
- RTE_SET_USED(conf_sz);
-
header->cn9k.pt = DPI_HDR_PT_ZBW_CA;
switch (conf->direction) {
@@ -163,54 +153,11 @@ cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t
vchan,
header->cn9k.fport = conf->dst_port.pcie.coreid;
header->cn9k.pvfe = 0;
};
-
- /* Free up descriptor memory before allocating. */
- cnxk_dmadev_vchan_free(dpivf, vchan);
-
- max_desc = conf->nb_desc;
- if (!rte_is_power_of_2(max_desc))
- max_desc = rte_align32pow2(max_desc);
-
- if (max_desc > CNXK_DPI_MAX_DESC)
- max_desc = CNXK_DPI_MAX_DESC;
-
- size = (max_desc * sizeof(struct cnxk_dpi_compl_s *));
- dpi_conf->c_desc.compl_ptr = rte_zmalloc(NULL, size, 0);
-
- if (dpi_conf->c_desc.compl_ptr == NULL) {
- plt_err("Failed to allocate for comp_data");
- return -ENOMEM;
- }
-
- for (i = 0; i < max_desc; i++) {
- dpi_conf->c_desc.compl_ptr[i] =
- rte_zmalloc(NULL, sizeof(struct cnxk_dpi_compl_s), 0);
- if (!dpi_conf->c_desc.compl_ptr[i]) {
- plt_err("Failed to allocate for descriptor memory");
- return -ENOMEM;
- }
-
- dpi_conf->c_desc.compl_ptr[i]->cdata = CNXK_DPI_REQ_CDATA;
- }
-
- dpi_conf->c_desc.max_cnt = (max_desc - 1);
-
- return 0;
}
-static int
-cn10k_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
-const struct rte_dma_vchan_conf *conf, uint32_t
conf_sz)
+static void
+cn10k_dmadev_setup_hdr(union cnxk_dpi_instr_cmd *header, const struct
rte_dma_vchan_conf *conf)
{
- struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
- struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
- union dpi_instr_hdr_s *header = &dpi_conf->hdr;
- uint16_t max_desc;
- uint32_t size;
- int i;
-
- RTE_SET_USED(conf_sz);
-
header->cn10k.pt = DPI_HDR_PT_ZBW_CA;
switch (conf->direction) {
@@ -246,6 +193,27 @@ cn10k_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t
vchan,
header->cn10k.fport = conf->dst_port.pcie.coreid;
header->cn10k.pvfe = 0;
};
+}
+
+static int
+cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
+ const struct rte_dma_vchan_conf *conf, uint32_t conf_sz)
+{
+ struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
+ struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
+ union cnxk_dpi_instr_cmd *header;
+ uint16_t max_desc;
+ uint32_t size;
+ int i;
+
+ RTE_SET_USED(conf_sz);
+
+ header = (union cnxk_dpi_instr_cmd *)&dpi_conf->cmd.u;
+
+ if (dpivf->is_cn10k)
+ cn10k_dmadev_setup_hdr(header, conf);
+ else
+ cn9k_dmadev_setup_hdr(header, conf);
/* Free up descriptor memory before allocating. */
cnxk_dmadev_vchan_free(dpivf, vchan);
@@ -371,333 +339,6 @@ cnxk_dmadev_close(struct rte_dma_dev *dev)
return 0;
}
-static inline int
-__dpi_queue_write(struct cnxk_dpi_vf_s *dpi, uint64_t *cmds, int cmd_count)
-{
- uint64_t *ptr = dpi->chunk_base;
-
- if ((cmd_count < DPI_MIN_CMD_SIZE) || (cmd_count > DPI_MAX_CMD_SIZE) ||
cmds == NULL)
- return -EINVAL;
-
- /*
-* Normally there is plenty of room in the current buffer for the
-* command
-*/
- if (dpi->chunk_head + cmd_count < dpi->chunk_size_m1) {
- ptr += dpi->chunk_head;
- dpi->chunk_head +=
[PATCH v8 1/2] dma/cnxk: rework DMA driver
From: Pavan Nikhilesh
To use the mempool cache, use rte_mempool for the DMA chunk pool.
Move the mempool creation to device start to limit the number of
chunks allocated based on the total number of descriptors
configured across all the vchans.
Remove unnecessary state tracking flags as the library handles it
and add the `CNXK` prefix to driver macros.
Convert the log register macro for all cnxk drivers to
RTE_LOG_REGISTER_DEFAULT.
Signed-off-by: Pavan Nikhilesh
---
Depends-on: 29324
v8 Changes:
- Actually include the fixes to checkpatch.
v7 Changes:
- Fix checkpatch warnings.
v6 Changes:
- Rework drvice configuration and start logic.
- add CNXK prefix to driver macros.
v5 Changes:
- Use RTE_LOG_REGISTER_DEFAULT for registering logging.
v4 Changes:
- Fix clang build.
v3 Changes:
- Fix build.
drivers/common/cnxk/roc_dpi.c | 90 ++-
drivers/common/cnxk/roc_dpi.h | 28 +---
drivers/common/cnxk/roc_dpi_priv.h | 3 -
drivers/common/cnxk/roc_platform.c | 21 +--
drivers/common/cnxk/roc_platform.h | 2 +
drivers/common/cnxk/version.map| 1 +
drivers/dma/cnxk/cnxk_dmadev.c | 252 -
drivers/dma/cnxk/cnxk_dmadev.h | 47 --
8 files changed, 205 insertions(+), 239 deletions(-)
diff --git a/drivers/common/cnxk/roc_dpi.c b/drivers/common/cnxk/roc_dpi.c
index 0e2f803077..c241168294 100644
--- a/drivers/common/cnxk/roc_dpi.c
+++ b/drivers/common/cnxk/roc_dpi.c
@@ -1,6 +1,7 @@
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright(C) 2021 Marvell.
*/
+
#include
#include
#include
@@ -52,17 +53,12 @@ roc_dpi_disable(struct roc_dpi *dpi)
}
int
-roc_dpi_configure(struct roc_dpi *roc_dpi)
+roc_dpi_configure(struct roc_dpi *roc_dpi, uint32_t chunk_sz, uint64_t aura,
uint64_t chunk_base)
{
struct plt_pci_device *pci_dev;
- const struct plt_memzone *dpi_mz;
dpi_mbox_msg_t mbox_msg;
- struct npa_pool_s pool;
- struct npa_aura_s aura;
- int rc, count, buflen;
- uint64_t aura_handle;
- plt_iova_t iova;
- char name[32];
+ uint64_t reg;
+ int rc;
if (!roc_dpi) {
plt_err("roc_dpi is NULL");
@@ -70,79 +66,30 @@ roc_dpi_configure(struct roc_dpi *roc_dpi)
}
pci_dev = roc_dpi->pci_dev;
- memset(&pool, 0, sizeof(struct npa_pool_s));
- pool.nat_align = 1;
-
- memset(&aura, 0, sizeof(aura));
- rc = roc_npa_pool_create(&aura_handle, DPI_CMD_QUEUE_SIZE,
-DPI_CMD_QUEUE_BUFS, &aura, &pool, 0);
- if (rc) {
- plt_err("Failed to create NPA pool, err %d\n", rc);
- return rc;
- }
-
- snprintf(name, sizeof(name), "dpimem%d:%d:%d:%d", pci_dev->addr.domain,
pci_dev->addr.bus,
-pci_dev->addr.devid, pci_dev->addr.function);
- buflen = DPI_CMD_QUEUE_SIZE * DPI_CMD_QUEUE_BUFS;
- dpi_mz = plt_memzone_reserve_aligned(name, buflen, 0,
DPI_CMD_QUEUE_SIZE);
- if (dpi_mz == NULL) {
- plt_err("dpi memzone reserve failed");
- rc = -ENOMEM;
- goto err1;
- }
-
- roc_dpi->mz = dpi_mz;
- iova = dpi_mz->iova;
- for (count = 0; count < DPI_CMD_QUEUE_BUFS; count++) {
- roc_npa_aura_op_free(aura_handle, 0, iova);
- iova += DPI_CMD_QUEUE_SIZE;
- }
-
- roc_dpi->chunk_base = (void *)roc_npa_aura_op_alloc(aura_handle, 0);
- if (!roc_dpi->chunk_base) {
- plt_err("Failed to alloc buffer from NPA aura");
- rc = -ENOMEM;
- goto err2;
- }
-
- roc_dpi->chunk_next = (void *)roc_npa_aura_op_alloc(aura_handle, 0);
- if (!roc_dpi->chunk_next) {
- plt_err("Failed to alloc buffer from NPA aura");
- rc = -ENOMEM;
- goto err2;
- }
- roc_dpi->aura_handle = aura_handle;
- /* subtract 2 as they have already been alloc'ed above */
- roc_dpi->pool_size_m1 = (DPI_CMD_QUEUE_SIZE >> 3) - 2;
+ roc_dpi_disable(roc_dpi);
+ reg = plt_read64(roc_dpi->rbase + DPI_VDMA_SADDR);
+ while (!(reg & BIT_ULL(63)))
+ reg = plt_read64(roc_dpi->rbase + DPI_VDMA_SADDR);
plt_write64(0x0, roc_dpi->rbase + DPI_VDMA_REQQ_CTL);
- plt_write64(((uint64_t)(roc_dpi->chunk_base) >> 7) << 7,
- roc_dpi->rbase + DPI_VDMA_SADDR);
+ plt_write64(chunk_base, roc_dpi->rbase + DPI_VDMA_SADDR);
mbox_msg.u[0] = 0;
mbox_msg.u[1] = 0;
/* DPI PF driver expects vfid starts from index 0 */
mbox_msg.s.vfid = roc_dpi->vfid;
mbox_msg.s.cmd = DPI_QUEUE_OPEN;
- mbox_msg.s.csize = DPI_CMD_QUEUE_SIZE;
- mbox_msg.s.aura = roc_npa_aura_handle_to_aura(aura_handle);
+ mbox_msg.s.csize = chunk_sz;
+ mbox_msg.s.aura = aura;
mbox_msg.s.sso_pf_func = idev_sso_pffunc_get();
mbox_msg.s.npa_pf_func = idev_npa_pffunc_get();
[PATCH v8 2/2] dma/cnxk: rewrite DMA fastpath
From: Pavan Nikhilesh
Rewrite DMA fastpath to use NEON instructions and reduce number
of words read from config.
Signed-off-by: Pavan Nikhilesh
---
drivers/dma/cnxk/cnxk_dmadev.c| 428 ++---
drivers/dma/cnxk/cnxk_dmadev.h| 59 +++-
drivers/dma/cnxk/cnxk_dmadev_fp.c | 436 ++
drivers/dma/cnxk/meson.build | 9 +-
4 files changed, 528 insertions(+), 404 deletions(-)
create mode 100644 drivers/dma/cnxk/cnxk_dmadev_fp.c
diff --git a/drivers/dma/cnxk/cnxk_dmadev.c b/drivers/dma/cnxk/cnxk_dmadev.c
index f58bb92dbc..26680edfde 100644
--- a/drivers/dma/cnxk/cnxk_dmadev.c
+++ b/drivers/dma/cnxk/cnxk_dmadev.c
@@ -115,19 +115,9 @@ cnxk_dmadev_configure(struct rte_dma_dev *dev, const
struct rte_dma_conf *conf,
return 0;
}
-static int
-cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
- const struct rte_dma_vchan_conf *conf, uint32_t conf_sz)
+static void
+cn9k_dmadev_setup_hdr(union cnxk_dpi_instr_cmd *header, const struct
rte_dma_vchan_conf *conf)
{
- struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
- struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
- union dpi_instr_hdr_s *header = &dpi_conf->hdr;
- uint16_t max_desc;
- uint32_t size;
- int i;
-
- RTE_SET_USED(conf_sz);
-
header->cn9k.pt = DPI_HDR_PT_ZBW_CA;
switch (conf->direction) {
@@ -163,54 +153,11 @@ cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t
vchan,
header->cn9k.fport = conf->dst_port.pcie.coreid;
header->cn9k.pvfe = 0;
};
-
- /* Free up descriptor memory before allocating. */
- cnxk_dmadev_vchan_free(dpivf, vchan);
-
- max_desc = conf->nb_desc;
- if (!rte_is_power_of_2(max_desc))
- max_desc = rte_align32pow2(max_desc);
-
- if (max_desc > CNXK_DPI_MAX_DESC)
- max_desc = CNXK_DPI_MAX_DESC;
-
- size = (max_desc * sizeof(struct cnxk_dpi_compl_s *));
- dpi_conf->c_desc.compl_ptr = rte_zmalloc(NULL, size, 0);
-
- if (dpi_conf->c_desc.compl_ptr == NULL) {
- plt_err("Failed to allocate for comp_data");
- return -ENOMEM;
- }
-
- for (i = 0; i < max_desc; i++) {
- dpi_conf->c_desc.compl_ptr[i] =
- rte_zmalloc(NULL, sizeof(struct cnxk_dpi_compl_s), 0);
- if (!dpi_conf->c_desc.compl_ptr[i]) {
- plt_err("Failed to allocate for descriptor memory");
- return -ENOMEM;
- }
-
- dpi_conf->c_desc.compl_ptr[i]->cdata = CNXK_DPI_REQ_CDATA;
- }
-
- dpi_conf->c_desc.max_cnt = (max_desc - 1);
-
- return 0;
}
-static int
-cn10k_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
-const struct rte_dma_vchan_conf *conf, uint32_t
conf_sz)
+static void
+cn10k_dmadev_setup_hdr(union cnxk_dpi_instr_cmd *header, const struct
rte_dma_vchan_conf *conf)
{
- struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
- struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
- union dpi_instr_hdr_s *header = &dpi_conf->hdr;
- uint16_t max_desc;
- uint32_t size;
- int i;
-
- RTE_SET_USED(conf_sz);
-
header->cn10k.pt = DPI_HDR_PT_ZBW_CA;
switch (conf->direction) {
@@ -246,6 +193,27 @@ cn10k_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t
vchan,
header->cn10k.fport = conf->dst_port.pcie.coreid;
header->cn10k.pvfe = 0;
};
+}
+
+static int
+cnxk_dmadev_vchan_setup(struct rte_dma_dev *dev, uint16_t vchan,
+ const struct rte_dma_vchan_conf *conf, uint32_t conf_sz)
+{
+ struct cnxk_dpi_vf_s *dpivf = dev->fp_obj->dev_private;
+ struct cnxk_dpi_conf *dpi_conf = &dpivf->conf[vchan];
+ union cnxk_dpi_instr_cmd *header;
+ uint16_t max_desc;
+ uint32_t size;
+ int i;
+
+ RTE_SET_USED(conf_sz);
+
+ header = (union cnxk_dpi_instr_cmd *)&dpi_conf->cmd.u;
+
+ if (dpivf->is_cn10k)
+ cn10k_dmadev_setup_hdr(header, conf);
+ else
+ cn9k_dmadev_setup_hdr(header, conf);
/* Free up descriptor memory before allocating. */
cnxk_dmadev_vchan_free(dpivf, vchan);
@@ -371,333 +339,6 @@ cnxk_dmadev_close(struct rte_dma_dev *dev)
return 0;
}
-static inline int
-__dpi_queue_write(struct cnxk_dpi_vf_s *dpi, uint64_t *cmds, int cmd_count)
-{
- uint64_t *ptr = dpi->chunk_base;
-
- if ((cmd_count < DPI_MIN_CMD_SIZE) || (cmd_count > DPI_MAX_CMD_SIZE) ||
cmds == NULL)
- return -EINVAL;
-
- /*
-* Normally there is plenty of room in the current buffer for the
-* command
-*/
- if (dpi->chunk_head + cmd_count < dpi->chunk_size_m1) {
- ptr += dpi->chunk_head;
- dpi->chunk_head +=
[PATCH 1/2] event/cnxk: flush flow context on cleanup
From: Pavan Nikhilesh
Flush currently held flow context on event port cleanup.
Signed-off-by: Pavan Nikhilesh
---
drivers/event/cnxk/cn10k_eventdev.c | 18 ++
drivers/event/cnxk/cn9k_eventdev.c | 25 +++--
2 files changed, 33 insertions(+), 10 deletions(-)
diff --git a/drivers/event/cnxk/cn10k_eventdev.c
b/drivers/event/cnxk/cn10k_eventdev.c
index 499a3aace7..211c51fd12 100644
--- a/drivers/event/cnxk/cn10k_eventdev.c
+++ b/drivers/event/cnxk/cn10k_eventdev.c
@@ -200,12 +200,14 @@ cn10k_sso_hws_reset(void *arg, void *hws)
cnxk_sso_hws_swtag_untag(base +
SSOW_LF_GWS_OP_SWTAG_UNTAG);
plt_write64(0, base + SSOW_LF_GWS_OP_DESCHED);
+ } else if (pend_tt != SSO_TT_EMPTY) {
+ plt_write64(0, base + SSOW_LF_GWS_OP_SWTAG_FLUSH);
}
/* Wait for desched to complete. */
do {
pend_state = plt_read64(base + SSOW_LF_GWS_PENDSTATE);
- } while (pend_state & BIT_ULL(58));
+ } while (pend_state & (BIT_ULL(58) | BIT_ULL(56)));
switch (dev->gw_mode) {
case CN10K_GW_MODE_PREF:
@@ -582,11 +584,16 @@ cn10k_sso_port_quiesce(struct rte_eventdev *event_dev,
void *port,
cn10k_sso_hws_get_work_empty(ws, &ev,
(NIX_RX_OFFLOAD_MAX - 1) | NIX_RX_REAS_F |
NIX_RX_MULTI_SEG_F);
- if (is_pend && ev.u64) {
+ if (is_pend && ev.u64)
if (flush_cb)
flush_cb(event_dev->data->dev_id, ev, args);
+ ptag = (plt_read64(ws->base + SSOW_LF_GWS_TAG) >> 32) & SSO_TT_EMPTY;
+ if (ptag != SSO_TT_EMPTY)
cnxk_sso_hws_swtag_flush(ws->base);
- }
+
+ do {
+ ptag = plt_read64(ws->base + SSOW_LF_GWS_PENDSTATE);
+ } while (ptag & BIT_ULL(56));
/* Check if we have work in PRF_WQE0, if so extract it. */
switch (dev->gw_mode) {
@@ -610,8 +617,11 @@ cn10k_sso_port_quiesce(struct rte_eventdev *event_dev,
void *port,
if (ev.u64) {
if (flush_cb)
flush_cb(event_dev->data->dev_id, ev, args);
- cnxk_sso_hws_swtag_flush(ws->base);
}
+ cnxk_sso_hws_swtag_flush(ws->base);
+ do {
+ ptag = plt_read64(ws->base + SSOW_LF_GWS_PENDSTATE);
+ } while (ptag & BIT_ULL(56));
}
ws->swtag_req = 0;
plt_write64(0, ws->base + SSOW_LF_GWS_OP_GWC_INVAL);
diff --git a/drivers/event/cnxk/cn9k_eventdev.c
b/drivers/event/cnxk/cn9k_eventdev.c
index 6cce5477f0..a03e3c138b 100644
--- a/drivers/event/cnxk/cn9k_eventdev.c
+++ b/drivers/event/cnxk/cn9k_eventdev.c
@@ -222,16 +222,16 @@ cn9k_sso_hws_reset(void *arg, void *hws)
cnxk_sso_hws_swtag_untag(
base + SSOW_LF_GWS_OP_SWTAG_UNTAG);
plt_write64(0, base + SSOW_LF_GWS_OP_DESCHED);
+ } else if (pend_tt != SSO_TT_EMPTY) {
+ plt_write64(0, base + SSOW_LF_GWS_OP_SWTAG_FLUSH);
}
/* Wait for desched to complete. */
do {
pend_state = plt_read64(base + SSOW_LF_GWS_PENDSTATE);
- } while (pend_state & BIT_ULL(58));
-
+ } while (pend_state & (BIT_ULL(58) | BIT_ULL(56)));
plt_write64(0, base + SSOW_LF_GWS_OP_GWC_INVAL);
}
-
if (dev->dual_ws)
dws->swtag_req = 0;
else
@@ -686,12 +686,25 @@ cn9k_sso_port_quiesce(struct rte_eventdev *event_dev,
void *port,
base, &ev, dev->rx_offloads,
dev->dual_ws ? dws->lookup_mem : ws->lookup_mem,
dev->dual_ws ? dws->tstamp : ws->tstamp);
- if (is_pend && ev.u64) {
+ if (is_pend && ev.u64)
if (flush_cb)
flush_cb(event_dev->data->dev_id, ev, args);
- cnxk_sso_hws_swtag_flush(ws->base);
- }
+
+ ptag = (plt_read64(base + SSOW_LF_GWS_TAG) >> 32) &
SSO_TT_EMPTY;
+ if (ptag != SSO_TT_EMPTY)
+ cnxk_sso_hws_swtag_flush(base);
+
+ do {
+ ptag = plt_read64(base + SSOW_LF_GWS_PENDSTATE);
+ } while (ptag & BIT_ULL(56));
+
+ plt_write64(0, base + SSOW_LF_GWS_OP_GWC_INVAL);
}
+
+ if (dev->dual_ws)
+ dws->swtag_req = 0;
+ else
+ ws->swtag_req = 0;
}
static int
--
2.25.1
[PATCH 2/2] common/cnxk: split XAQ counts
From: Pavan Nikhilesh Split XAQ counts into reserved and cached to allow more events to be inflight. Signed-off-by: Pavan Nikhilesh --- drivers/common/cnxk/roc_sso.c | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/drivers/common/cnxk/roc_sso.c b/drivers/common/cnxk/roc_sso.c index a5f48d5bbc..0a1074b018 100644 --- a/drivers/common/cnxk/roc_sso.c +++ b/drivers/common/cnxk/roc_sso.c @@ -5,7 +5,8 @@ #include "roc_api.h" #include "roc_priv.h" -#define SSO_XAQ_CACHE_CNT (0x7) +#define SSO_XAQ_CACHE_CNT (0x3) +#define SSO_XAQ_RSVD_CNT (0x4) #define SSO_XAQ_SLACK(16) /* Private functions. */ @@ -499,6 +500,7 @@ sso_hwgrp_init_xaq_aura(struct dev *dev, struct roc_sso_xaq_data *xaq, * pipelining. */ xaq->nb_xaq = (SSO_XAQ_CACHE_CNT * nb_hwgrp); + xaq->nb_xaq += (SSO_XAQ_RSVD_CNT * nb_hwgrp); xaq->nb_xaq += PLT_MAX(1 + ((xaq->nb_xae - 1) / xae_waes), xaq->nb_xaq); xaq->nb_xaq += SSO_XAQ_SLACK; @@ -542,8 +544,7 @@ sso_hwgrp_init_xaq_aura(struct dev *dev, struct roc_sso_xaq_data *xaq, * There should be a minimum headroom of 7 XAQs per HWGRP for SSO * to request XAQ to cache them even before enqueue is called. */ - xaq->xaq_lmt = - xaq->nb_xaq - (nb_hwgrp * SSO_XAQ_CACHE_CNT) - SSO_XAQ_SLACK; + xaq->xaq_lmt = xaq->nb_xaq - (nb_hwgrp * SSO_XAQ_CACHE_CNT) - SSO_XAQ_SLACK; return 0; npa_fill_fail: -- 2.25.1
RE: [PATCH 11/11] lib: remove pthread.h from includes
Hi, > -Original Message- > From: Thomas Monjalon > Sent: Thursday, September 7, 2023 12:12 AM > To: dev@dpdk.org > Cc: Tyler Retzlaff ; David Marchand > ; Ferruh Yigit ; Chas > Williams ; Min Hu (Connor) ; > Matan Azrad ; Viacheslav Ovsiienko > ; Ori Kam ; Suanming Mou > ; Ajit Khaparde ; > Somnath Kotur ; Devendra Singh Rawat > ; Alok Prasad ; Xu, Rosen > ; Zhang, Tianfei ; Vijay > Kumar Srivastava ; Jerin Jacob ; > Sunil Kumar Kori ; Burakov, Anatoly > ; Richardson, Bruce > ; Andrew Rybchenko > ; Naga Harish K, S V > ; Maxime Coquelin > ; Xia, Chenbo > Subject: [PATCH 11/11] lib: remove pthread.h from includes > > The header files should have the minimum embedded includes. > The file pthread.h can logically be removed from rte_per_lcore.h and > rte_ethdev_core.h files. > > Signed-off-by: Thomas Monjalon > --- > drivers/raw/ifpga/base/opae_osdep.h| 1 + Acked-by: Rosen Xu
RE: [PATCH 16/36] net/ipn3ke: fix Rx and Tx queue state
Hi,
> -Original Message-
> From: Jie Hai
> Sent: Friday, September 8, 2023 7:29 PM
> To: dev@dpdk.org; Xu, Rosen ; Ferruh Yigit
> ; Lijun Ou ; Chengwen Feng
> ; Konstantin Ananyev
> <"konstantin.v.ananyev@yandex.rukonstantin.ananyev"@huawei.com>;
> Thomas Monjalon
> Cc: haij...@huawei.com; lihuis...@huawei.com
> Subject: [PATCH 16/36] net/ipn3ke: fix Rx and Tx queue state
>
> The DPDK framework reports the queue state, which is stored in
> dev->data->tx_queue_state and dev->data->rx_queue_state. The
> state is maintained by the driver. Users may determine whether a queue
> participates in packet forwarding based on the state.
> Therefore, the driver needs to modify the queue state in time according to
> the actual situation.
>
> Fixes: 9ad9ff476cac ("ethdev: add queue state in queried queue
> information")
> Cc: sta...@dpdk.org
>
> Signed-off-by: Jie Hai
> ---
> drivers/net/ipn3ke/ipn3ke_representor.c | 12
> 1 file changed, 12 insertions(+)
>
> diff --git a/drivers/net/ipn3ke/ipn3ke_representor.c
> b/drivers/net/ipn3ke/ipn3ke_representor.c
> index c82f8b533370..d904d3f251e3 100644
> --- a/drivers/net/ipn3ke/ipn3ke_representor.c
> +++ b/drivers/net/ipn3ke/ipn3ke_representor.c
> @@ -120,6 +120,7 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
> uint64_t base_mac;
> uint32_t val;
> char attr_name[IPN3KE_RAWDEV_ATTR_LEN_MAX];
> + uint16_t i;
>
> rawdev = hw->rawdev;
>
> @@ -190,6 +191,11 @@ ipn3ke_rpst_dev_start(struct rte_eth_dev *dev)
>
> ipn3ke_rpst_link_update(dev, 0);
>
> + for (i = 0; i < dev->data->nb_rx_queues; i++)
> + dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> + for (i = 0; i < dev->data->nb_tx_queues; i++)
> + dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STARTED;
> +
> return 0;
> }
>
> @@ -198,6 +204,7 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev) {
> struct ipn3ke_hw *hw = IPN3KE_DEV_PRIVATE_TO_HW(dev);
> struct ipn3ke_rpst *rpst = IPN3KE_DEV_PRIVATE_TO_RPST(dev);
> + uint16_t i;
>
> if (hw->retimer.mac_type ==
> IFPGA_RAWDEV_RETIMER_MAC_TYPE_10GE_XFI) {
> /* Disable the TX path */
> @@ -207,6 +214,11 @@ ipn3ke_rpst_dev_stop(struct rte_eth_dev *dev)
> ipn3ke_xmac_rx_disable(hw, rpst->port_id, 0);
> }
>
> + for (i = 0; i < dev->data->nb_rx_queues; i++)
> + dev->data->rx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> + for (i = 0; i < dev->data->nb_tx_queues; i++)
> + dev->data->tx_queue_state[i] =
> RTE_ETH_QUEUE_STATE_STOPPED;
> +
> return 0;
> }
>
> --
> 2.30.0
Reviewed-by: Rosen Xu
