On 2024-04-23 13:15, Ferruh Yigit wrote:
On 4/23/2024 10:08 AM, Mattias Rönnblom wrote:
Cache align Rx and Tx queue struct to avoid false sharing.
RX struct happens to be 64 bytes on x86_64 already, so cache alignment
makes no change there, but it does on 32-bit ISAs.
TX struct is 56 bytes on x86_64.
Hi Mattias,
No objection to the patch. Is the improvement theoretical or do you
measure any improvement practically, if so how much is the improvement?
I didn't run any benchmarks.
Two cores storing to a (falsely) shared cache line on a per-packet basis
is going to be very expensive, at least for "light touch" applications.
Both structs keep counters, and in the RX case they are updated even
for empty polls.
Do you think does it help if move 'rx_pkts' & 'rx_bytes' update within
the loop?
No, why? Wouldn't that be worse? Especially since rx_pkts and rx_bytes
are declared volatile, so you are forcing a load-modify-store cycle for
every increment.
I would drop "volatile", or replace it with an atomic (although *not*
use an atomic add for incrementing, but rather atomic load + <n>
non-atomic adds + atomic store).
Signed-off-by: Mattias Rönnblom <mattias.ronnb...@ericsson.com>
---
drivers/net/af_packet/rte_eth_af_packet.c | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/net/af_packet/rte_eth_af_packet.c
b/drivers/net/af_packet/rte_eth_af_packet.c
index 397a32db58..28aeb7d08e 100644
--- a/drivers/net/af_packet/rte_eth_af_packet.c
+++ b/drivers/net/af_packet/rte_eth_af_packet.c
@@ -6,6 +6,7 @@
* All rights reserved.
*/
+#include <rte_common.h>
#include <rte_string_fns.h>
#include <rte_mbuf.h>
#include <ethdev_driver.h>
@@ -53,7 +54,7 @@ struct pkt_rx_queue {
volatile unsigned long rx_pkts;
volatile unsigned long rx_bytes;
-};
+} __rte_cache_aligned;
Latest location for '__rte_cache_aligned' tag is at the beginning of the
struct [1], so something like:
`struct __rte_cache_aligned pkt_rx_queue {`
[1]
https://patchwork.dpdk.org/project/dpdk/list/?series=31746&state=%2A&archive=both
