> From: Jerin Jacob [mailto:jerinjac...@gmail.com] > Sent: Wednesday, 21 February 2024 10.44 > > On Tue, Feb 20, 2024 at 2:35 PM Mattias Rönnblom > <mattias.ronnb...@ericsson.com> wrote: > > > > Introduce DPDK per-lcore id variables, or lcore variables for short. > > > > An lcore variable has one value for every current and future lcore > > id-equipped thread. > > > > The primary <rte_lcore_var.h> use case is for statically allocating > > small chunks of often-used data, which is related logically, but > where > > there are performance benefits to reap from having updates being > local > > to an lcore. > > I think, in order to quantify the gain, we must add a performance test > case to measure the acces cycles with lcore variables scheme vs this > scheme. > Other PMU counters(Cache misses) may be interesting but we dont have > means in DPDK to do self monitoring now like > https://patches.dpdk.org/project/dpdk/patch/20221213104350.3218167-1- > tduszyn...@marvell.com/ > > > > > Lcore variables are similar to thread-local storage (TLS, e.g., C11 > > _Thread_local), but decoupling the values' life time with that of the > > threads.
Lcore variables can be accessed by other threads, unlike TLS variables. If a TLS variable needs to be accessed by other threads, there must also be an RTE_MAX_LCORE-sized array of pointers to the TLS variable, where each worker thread must initialize the entry pointing to its TLS variable. > > > > Lcore variables are also similar in terms of functionality provided > by > > FreeBSD kernel's DPCPU_*() family of macros and the associated > > build-time machinery. DPCPU uses linker scripts, which effectively > > prevents the reuse of its, otherwise seemingly viable, approach. > > > > The currently-prevailing way to solve the same problem as lcore > > variables is to keep a module's per-lcore data as RTE_MAX_LCORE-sized > > array of cache-aligned, RTE_CACHE_GUARDed structs. The benefit of > > lcore variables over this approach is that data related to the same > > lcore now is close (spatially, in memory), rather than data used by > > the same module, which in turn avoid excessive use of padding, > > polluting caches with unused data. > > There are 3 ways to implement per-lcore variables: 1. Thread-local storage, available via RTE_DEFINE_PER_LCORE(type, name). 2. RTE_MAX_LCORE-sized arrays. 3. Lcore variables, as provided by this patch series. Perhaps an overview of differences and performance numbers would help understand the benefits of this patch series. The advantages of packing more variables into the same cache line may be hard to measure without PMU counters, and could perhaps be described or estimated instead.
