On Thu, Mar 21, 2024 at 10:32:02AM +0000, Bruce Richardson wrote:
> On Wed, Mar 20, 2024 at 03:01:36PM -0700, Tyler Retzlaff wrote:
> > RTE_MARKER typedefs are a GCC extension unsupported by MSVC. Remove
> > RTE_MARKER fields from rte_mbuf struct.
> >
> > Maintain alignment of fields after removed cacheline1 marker by placing
> > C11 alignas(RTE_CACHE_LINE_MIN_SIZE).
> >
> > Provide new rearm_data and rx_descriptor_fields1 fields in anonymous
> > unions as single element arrays of with types matching the original
> > markers to maintain API compatibility.
> >
> > Signed-off-by: Tyler Retzlaff <roret...@linux.microsoft.com>
> > ---
> > doc/guides/rel_notes/release_24_03.rst | 2 +
> > lib/mbuf/rte_mbuf.h | 4 +-
> > lib/mbuf/rte_mbuf_core.h | 188
> > ++++++++++++++++++---------------
> > 3 files changed, 104 insertions(+), 90 deletions(-)
> >
> > diff --git a/doc/guides/rel_notes/release_24_03.rst
> > b/doc/guides/rel_notes/release_24_03.rst
> > index 14826ea..4f18cca 100644
> > --- a/doc/guides/rel_notes/release_24_03.rst
> > +++ b/doc/guides/rel_notes/release_24_03.rst
> > @@ -216,6 +216,8 @@ Removed Items
> >
> > * acc101: Removed obsolete code for non productized HW variant.
> >
> > +* mbuf: ``RTE_MARKER`` fields ``cacheline0`` and ``cacheline1``
> > + have been removed from ``struct rte_mbuf``.
> >
> > API Changes
> > -----------
> > diff --git a/lib/mbuf/rte_mbuf.h b/lib/mbuf/rte_mbuf.h
> > index 286b32b..4c4722e 100644
> > --- a/lib/mbuf/rte_mbuf.h
> > +++ b/lib/mbuf/rte_mbuf.h
> > @@ -108,7 +108,7 @@
> > static inline void
> > rte_mbuf_prefetch_part1(struct rte_mbuf *m)
> > {
> > - rte_prefetch0(&m->cacheline0);
> > + rte_prefetch0(m);
> > }
> >
> > /**
> > @@ -126,7 +126,7 @@
> > rte_mbuf_prefetch_part2(struct rte_mbuf *m)
> > {
> > #if RTE_CACHE_LINE_SIZE == 64
> > - rte_prefetch0(&m->cacheline1);
> > + rte_prefetch0(RTE_PTR_ADD(m, RTE_CACHE_LINE_MIN_SIZE));
> > #else
> > RTE_SET_USED(m);
> > #endif
> > diff --git a/lib/mbuf/rte_mbuf_core.h b/lib/mbuf/rte_mbuf_core.h
> > index 9f58076..665213c 100644
> > --- a/lib/mbuf/rte_mbuf_core.h
> > +++ b/lib/mbuf/rte_mbuf_core.h
> > @@ -465,8 +465,6 @@ enum {
> > * The generic rte_mbuf, containing a packet mbuf.
> > */
> > struct __rte_cache_aligned rte_mbuf {
> > - RTE_MARKER cacheline0;
> > -
> > void *buf_addr; /**< Virtual address of segment buffer. */
> > #if RTE_IOVA_IN_MBUF
> > /**
> > @@ -488,116 +486,130 @@ struct __rte_cache_aligned rte_mbuf {
> > #endif
> >
> > /* next 8 bytes are initialised on RX descriptor rearm */
> > - RTE_MARKER64 rearm_data;
> > - uint16_t data_off;
> > -
> > - /**
> > - * Reference counter. Its size should at least equal to the size
> > - * of port field (16 bits), to support zero-copy broadcast.
> > - * It should only be accessed using the following functions:
> > - * rte_mbuf_refcnt_update(), rte_mbuf_refcnt_read(), and
> > - * rte_mbuf_refcnt_set(). The functionality of these functions (atomic,
> > - * or non-atomic) is controlled by the RTE_MBUF_REFCNT_ATOMIC flag.
> > - */
> > - RTE_ATOMIC(uint16_t) refcnt;
> > + union {
> > + uint64_t rearm_data[1];
> > + __extension__
> > + struct {
> > + uint16_t data_off;
> > +
> > + /**
> > + * Reference counter. Its size should at least equal to
> > the size
> > + * of port field (16 bits), to support zero-copy
> > broadcast.
> > + * It should only be accessed using the following
> > functions:
> > + * rte_mbuf_refcnt_update(), rte_mbuf_refcnt_read(), and
> > + * rte_mbuf_refcnt_set(). The functionality of these
> > functions (atomic,
> > + * or non-atomic) is controlled by the
> > RTE_MBUF_REFCNT_ATOMIC flag.
> > + */
> > + RTE_ATOMIC(uint16_t) refcnt;
> >
> > - /**
> > - * Number of segments. Only valid for the first segment of an mbuf
> > - * chain.
> > - */
> > - uint16_t nb_segs;
> > + /**
> > + * Number of segments. Only valid for the first segment
> > of an mbuf
> > + * chain.
> > + */
> > + uint16_t nb_segs;
> >
> > - /** Input port (16 bits to support more than 256 virtual ports).
> > - * The event eth Tx adapter uses this field to specify the output port.
> > - */
> > - uint16_t port;
> > + /** Input port (16 bits to support more than 256
> > virtual ports).
> > + * The event eth Tx adapter uses this field to specify
> > the output port.
> > + */
> > + uint16_t port;
> > + };
> > + };
> >
> > uint64_t ol_flags; /**< Offload features. */
> >
> > /* remaining bytes are set on RX when pulling packet from descriptor */
> > - RTE_MARKER rx_descriptor_fields1;
> > -
> > - /*
> > - * The packet type, which is the combination of outer/inner L2, L3, L4
> > - * and tunnel types. The packet_type is about data really present in the
> > - * mbuf. Example: if vlan stripping is enabled, a received vlan packet
> > - * would have RTE_PTYPE_L2_ETHER and not RTE_PTYPE_L2_VLAN because the
> > - * vlan is stripped from the data.
> > - */
> > union {
> > - uint32_t packet_type; /**< L2/L3/L4 and tunnel information. */
> > + void *rx_descriptor_fields1[1];
>
> Can we make this array the actual size of all the fields, rather than just
> an 8-byte value? That would allow the right think to be done if assigning
> the descriptor fields from one mbuf to another, or when using memset or
> memcpy on them.
Morten pointed out in a previous version that the marker being an array of
void * was a bug to begin with.
The other field of the union is 24 bytes. I suppose it would be possible
to conditionally compile the array to be either 3 or 6 elements. I guess
this would be an improvement over what the marker is doing now.
Just a reminder that we cannot 'correct' the type since that would
require adaptation of calling code.
What do others think? Keep it as a single element array or conditional
compile based on sizeof(void *)?
>
> /Bruce
