> From: Konstantin Ananyev [mailto:[email protected]] > Sent: Monday, 31 August 2026 11.38 > > > > From: Konstantin Ananyev [mailto:[email protected]] > > > Sent: Monday, 31 August 2026 10.50 > > > > > > > Added a new high-performance lock-free "pile", using the Stack > API. > > > > The pile behaves roughly like a stack, but is not strictly LIFO. > > > > > > > > The pile is optimized for pushing/popping bulks of objects, which > > > > it does significantly faster than the lock-free stack. > > > > > > > > Pushing/popping a number of objects not divisible by the compile > time > > > > configurable bulk size is handled gracefully, but not as fast as > > > > complete bulks. > > > > > > > > Performance examples, stack_pile_perf_autotest vs. > stack_lf_autotest: > > > > > > > > On a single core, pushing/popping 1 or 8 objects is similar > speed. > > > > On a single core, pushing/popping 32 objects is 2x faster. > > > > On a single core, pushing/popping 512 objects is 10x faster. > > > > > > > > On four cores, pushing/popping 1, 8 or 32 objects is slightly > faster. > > > > On four cores, pushing/popping 512 objects is 4x faster. > > > > > > Acked-by: Konstantin Ananyev <[email protected]> > > > > > > The code itself looks ok to me, thought I still think it is worth > to > > > consider > > > moving lf_pile (and lf_stack) DP implementation in .c, to avoid > each > > > rte_stack_pus/pop > > > to inline all three of them. > > > > If a use case knows the selected stack implement at build time, it > can call the > > implementation's push/pop functions directly. > > I updated the mempool stack driver v3 patch [1] to do this for all > three stack > > implementations. > > That's good thing for sure. > Though I am talking about: > #ifdef __cplusplus > extern "C" { > @@ -115,6 +174,8 @@ rte_stack_push(struct rte_stack *s, void * const > *obj_table, unsigned int n) > > if (s->flags & RTE_STACK_F_LF) > return __rte_stack_lf_push(s, obj_table, n); > + else if (s->flags & RTE_STACK_F_PILE) > + return __rte_stack_pile_push(s, obj_table, n); > else > return __rte_stack_std_push(s, obj_table, n); > } > @@ -139,6 +200,8 @@ rte_stack_pop(struct rte_stack *s, void > **obj_table, unsigned int n) > > if (s->flags & RTE_STACK_F_LF) > return __rte_stack_lf_pop(s, obj_table, n); > + else if (s->flags & RTE_STACK_F_PILE) > + return __rte_stack_pile_pop(s, obj_table, n); > else > return __rte_stack_std_pop(s, obj_table, n); > } > > In rte_stack.h > We still can have our __rte_stack_pile_push/po as inline functions in > the internal headers, > so mempool (and whoever else needs them) can include them directly. > My suggestion to have rte_stack_pile_pop() in .c and invoke it (not- > inlined one) from > generic rte_stack_pop().
I get it, and I agree it would reduce the footprint of the compiled code. But it would eliminate compiler optimizations for build time known sizes, e.g. n=1 or n=BURST_SIZE, so I prefer keeping them inlined. De-inlining could also ruin branch prediction when used with multiple stacks. Let's say the same number of objects is always dequeued from a specific stack; the branch predictor would learn this. Now, if the non-inlined function is called to dequeue a different number of objects from another stack, it would trip up the branch predictor. BTW, The rings use the same design pattern, where the enqueue/dequeue implementation is selected at run-time based on ring->prod/cons.sync_type, all inlined for the benefit of the optimizer when various parameters (element size, number of objects) are known at build time, and for the benefit of the branch predictor. > > > [1]: > https://patchwork.dpdk.org/project/dpdk/patch/20260827135556.522443-3- > > [email protected]/ > > > > > My speculation is that the perf diff for bulk enqueue/dequeue > because > > > of > > > that would be negligible, while both are quite big for inlining > them > > > always > > > (specially lf_pile). > > > > If the number of objects being pushed/popped is known at build time, > the > > compiler can optimize the functions when inlined. > > > > I tried experimented with conditional inlining depending on the > number of > > objects being known at build time, but I wasn't really pleased with > it.
