On Tue, Sep 17, 2024 at 5:49 PM Konstantin Ananyev
<konstantin.v.anan...@yandex.ru> wrote:
>
> From: Konstantin Ananyev <konstantin.anan...@huawei.com>
>
> Staged-Ordered-Ring (SORING) provides a SW abstraction for 'ordered' queues
> with multiple processing 'stages'.
> It is based on conventional DPDK rte_ring, re-uses many of its concepts,
> and even substantial part of its code.
> It can be viewed as an 'extension' of rte_ring functionality.
> In particular, main SORING properties:
> - circular ring buffer with fixed size objects
> - producer, consumer plus multiple processing stages in the middle.
> - allows to split objects processing into multiple stages.
> - objects remain in the same ring while moving from one stage to the other,
> initial order is preserved, no extra copying needed.
> - preserves the ingress order of objects within the queue across multiple
> stages, i.e.:
> at the same stage multiple threads can process objects from the ring in
> any order, but for the next stage objects will always appear in the
> original order.
> - each stage (and producer/consumer) can be served by single and/or
> multiple threads.
> - number of stages, size and number of objects in the ring are
> configurable at ring initialization time.
>
> Data-path API provides four main operations:
> - enqueue/dequeue works in the same manner as for conventional rte_ring,
> all rte_ring synchronization types are supported.
> - acquire/release - for each stage there is an acquire (start) and
> release (finish) operation.
> after some objects are 'acquired' - given thread can safely assume that
> it has exclusive possession of these objects till 'release' for them is
> invoked.
> Note that right now user has to release exactly the same number of
> objects that was acquired before.
> After 'release', objects can be 'acquired' by next stage and/or dequeued
> by the consumer (in case of last stage).
>
> Expected use-case: applications that uses pipeline model
> (probably with multiple stages) for packet processing, when preserving
> incoming packet order is important. I.E.: IPsec processing, etc.
>
> Signed-off-by: Eimear Morrissey <eimear.morris...@huawei.com>
> Signed-off-by: Konstantin Ananyev <konstantin.anan...@huawei.com>
> ---
> doc/guides/rel_notes/release_24_11.rst | 8 +
> lib/ring/meson.build | 4 +-
> lib/ring/rte_soring.c | 182 ++++++++
> lib/ring/rte_soring.h | 543 ++++++++++++++++++++++++
> lib/ring/soring.c | 548 +++++++++++++++++++++++++
> lib/ring/soring.h | 124 ++++++
> lib/ring/version.map | 19 +
> 7 files changed, 1426 insertions(+), 2 deletions(-)
> create mode 100644 lib/ring/rte_soring.c
> create mode 100644 lib/ring/rte_soring.h
> create mode 100644 lib/ring/soring.c
> create mode 100644 lib/ring/soring.h
Good feature and makes sense to not make as eventdev driver.
# I think, it is worth updating doc/guides/prog_guide/ring_lib.rst for
this new type and means to use it.
# Missing rte_soring.h update in doc/api/doxy-api-index.md
> diff --git a/lib/ring/rte_soring.h b/lib/ring/rte_soring.h
> new file mode 100644
> index 0000000000..9c35b4a18c
> --- /dev/null
> +++ b/lib/ring/rte_soring.h
> @@ -0,0 +1,543 @@
> +/* SPDX-License-Identifier: BSD-3-Clause
> + * Copyright(c) 2024 Huawei Technologies Co., Ltd
> + */
> +
> +#ifndef _RTE_SORING_H_
> +#define _RTE_SORING_H_
> +
> +/**
> + * @file
> + * This file contains definition of RTE soring (Staged Ordered Ring) public
> API.
> + * Brief description:
> + * enqueue/dequeue works the same as for conventional rte_ring:
> + * any rte_ring sync types can be used, etc.
> + * Plus there could be multiple 'stages'.
> + * For each stage there is an acquire (start) and release (finish) operation.
> + * after some elems are 'acquired' - user can safely assume that he has
> + * exclusive possession of these elems till 'release' for them is done.
> + * Note that right now user has to release exactly the same number of elems
> + * he acquired before.
> + * After 'release', elems can be 'acquired' by next stage and/or dequeued
> + * (in case of last stage).
> + */
> +
> +#ifdef __cplusplus
> +extern "C" {
> +#endif
> +
> +#include <rte_ring.h>
> +
> +/* upper 2 bits are used for status */
Add Doxygen comment for public symbol
> +#define RTE_SORING_ST_BIT 30
> +
> +/* max possible number of elements in the soring */
Add Doxygen comment for public symbol
> +#define RTE_SORING_ELEM_MAX (RTE_BIT32(RTE_SORING_ST_BIT) - 1)
> +
> +struct rte_soring_param {
> + /** expected name of the ring */
> + const char *name;
> + /** number of elemnts in the ring */
> + uint32_t elems;
> + /** size of elements in the ring, must be a multiple of 4 */
> + uint32_t elem_size;
> + /**
> + * size of metadata for each elem, must be a multiple of 4.
> + * This parameter defines a size of supplementary and optional
> + * array of metadata associated with each object in the soring.
> + * While element size is configurable (see 'elem_size' parameter
> above),
> + * so user can specify it big enough to hold both object and its
> + * metadata together, for performance reasons it might be plausible
> + * to access them as separate arrays.
> + * Common usage scenario when such separation helps:
> + * enqueue() - writes to objects array
> + * acquire() - reads from objects array
> + * release() - writes to metadata array (as an example: return code)
> + * dequeue() - reads both objects and metadata array
> + */
> + uint32_t meta_size;
> + /** number of stages in the ring */
> + uint32_t stages;
> + /** sync type for producer */
> + enum rte_ring_sync_type prod_synt;
> + /** sync type for consumer */
> + enum rte_ring_sync_type cons_synt;
> +};
> +
> +struct rte_soring;
> +
> +/**
> + * Calculate the memory size needed for a soring
> + *
> + * This function returns the number of bytes needed for a ring, given
> + * the expected parameters for it. This value is the sum of the size of
> + * the internal metadata and the size of the memory needed by the
> + * actual ring elements and theri rec-codes. The value is aligned to a cache
typo - theri
> + * line size.
> + *
> + * @param prm
> + * Pointer to the structure that contains soring creation paramers.
> + * @return
> + * - The memory size needed for the soring on success.
> + * - -EINVAL if provided paramer values are invalid.
> + */
> +__rte_experimental
> +ssize_t
> +rte_soring_get_memsize(const struct rte_soring_param *prm);
> +
> +/**
> + * Initialize a soring structure.
> + *
> + * Initialize a soring structure in memory pointed by "r".
> + * The size of the memory area must be large enough to store the soring
> + * internal structures plus the objects and ret-code tables.
typo- ret-code
> + * It is strongly advised to use rte_ring_get_memsize() to get the
> + * appropriate size.
> + *
> + * @param r
> + * Pointer to the soring structure.
I think, you can also mention, memory allocated with size provided by
rte_soring_get_memsize().
Also add @see to cross-reference.
> + * @param prm
> + * Pointer to the structure that contains soring creation paramers.
> + * @return
> + * - 0 on success, or a negative error code.
> + */
> +__rte_experimental
> +int
> +rte_soring_init(struct rte_soring *r, const struct rte_soring_param *prm);
> +
> +/**
> + * Return the total number of filled entries in a ring.
Across the patch, a ring can be replaced by a soring ring
> + *
> + * @param r
> + * A pointer to the soring structure.
> + * @return
> + * The number of entries in the ring.
> + */
> +__rte_experimental
> +unsigned int
> +rte_soring_count(const struct rte_soring *r);
> +
> +/**
> + * Return the total number of unfilled entries in a ring.
> + *
> + * @param r
> + * A pointer to the soring structure.
> + * @return
> + * The number of free entries in the ring.
> + */
> +__rte_experimental
> +unsigned int
> +rte_soring_free_count(const struct rte_soring *r);
> +
> +/**
> + * Dump the status of the soring
> + *
> + * @param f
> + * A pointer to a file for output
> + * @param r
> + * Pointer to the soring structure.
> + */
> +__rte_experimental
> +void
> +rte_soring_dump(FILE *f, const struct rte_soring *r);
> +
> +/**
> + * Enqueue several objects on the ring.
> + *
> + * @param r
> + * A pointer to the soring structure.
> + * @param objs
> + * A pointer to an array of objects to enqueue.
> + * Size of objects to enqueue must be the same value as 'elem_size'
> parameter
> + * used while creating the ring. Otherwise the results are undefined.
> + * @param n
> + * The number of objects to add in the ring from the 'objs'.
> + * @param free_space
> + * if non-NULL, returns the amount of space in the ring after the
> + * enqueue operation has finished.
> + * @return
> + * - Actual number of objects enqueued, either 0 or @n.
> + */
> +__rte_experimental
> +uint32_t
> +rte_soring_enqueue_bulk(struct rte_soring *r, const void *objs,
> + uint32_t n, uint32_t *free_space);
> +
> +/**
> + * Enqueue several objects plus metadata on the ring.
> + *
> + * @param r
> + * A pointer to the soring structure.
> + * @param objs
> + * A pointer to an array of objects to enqueue.
> + * Size of objects to enqueue must be the same value as 'elem_size'
> parameter
> + * used while creating the ring. Otherwise the results are undefined.
> + * @param meta
> + * A pointer to an array of metadata values for each object to enqueue.
> + * Note that if user not using object metadata values, then this parameter
> + * can be NULL.
> + * Size of elements in this array must be the same value as 'meta_size'
> + * parameter used while creating the ring. If user created the soring with
> + * 'meta_size' value equals zero, then 'meta' parameter should be NULL.
> + * Otherwise the results are undefined.
> + * @param n
> + * The number of objects to add in the ring from the 'objs'.
> + * @param free_space
> + * if non-NULL, returns the amount of space in the ring after the
> + * enqueue operation has finished.
> + * @return
> + * - Actual number of objects enqueued, either 0 or @n.
> + */
> +__rte_experimental
> +uint32_t
> +rte_soring_enqueux_bulk(struct rte_soring *r, const void *objs,
> + const void *meta, uint32_t n, uint32_t *free_space);
> +
> +/**
> + * Enqueue several objects on the ring.
Across the patch, _burst and _bulk has the same API comment
> + *
> + * @param r
> + * A pointer to the soring structure.
> + * @param objs
> + * A pointer to an array of objects to enqueue.
> + * Size of objects to enqueue must be the same value as 'elem_size'
> parameter
> + * used while creating the ring. Otherwise the results are undefined.
> + * @param n
> + * The number of objects to add in the ring from the 'objs'.
> + * @param free_space
> + * if non-NULL, returns the amount of space in the ring after the
> + * enqueue operation has finished.
> + * @return
> + * - Actual number of objects enqueued.
> + */
> +__rte_experimental
> +uint32_t
> +rte_soring_enqueue_burst(struct rte_soring *r, const void *objs,
> + uint32_t n, uint32_t *free_space);
> +
> +/**
> + * Acquire several objects from the ring for given stage.
> + *
> + * @param r
> + * A pointer to the soring structure.
> + * @param objs
> + * A pointer to an array of objects to acquire.
> + * Size of objects must be the same value as 'elem_size' parameter
> + * used while creating the ring. Otherwise the results are undefined.
> + * @param stage
> + * Stage to acquire objects for.
> + * @param num
> + * The number of objects to acquire.
> + * @param ftoken
> + * Pointer to the opaque 'token' value used by release() op.
> + * User has to store this value somewhere, and later provide to the
> + * release().
> + * @param available
> + * If non-NULL, returns the number of remaining ring entries for given
> stage
> + * after the acquire has finished.
> + * @return
> + * - Actual number of objects acquired, either 0 or 'num'.
> + */
> +__rte_experimental
> +uint32_t
> +rte_soring_acquire_bulk(struct rte_soring *r, void *objs,
> + uint32_t stage, uint32_t num, uint32_t *ftoken, uint32_t *available);
Does stage needs to be uint32_t?
