On 2022-07-08 18:21, Bruce Richardson wrote:
On Fri, Jul 08, 2022 at 03:31:01PM +0000, Van Haaren, Harry wrote:
-----Original Message-----
From: Honnappa Nagarahalli <honnappa.nagaraha...@arm.com>
Sent: Friday, July 8, 2022 4:16 PM
To: Van Haaren, Harry <harry.van.haa...@intel.com>; dev@dpdk.org
Cc: mattias.ronnblom <mattias.ronnb...@ericsson.com>; Morten Brørup
<m...@smartsharesystems.com>; nd <n...@arm.com>; nd <n...@arm.com>
Subject: RE: [PATCH 2/2] service: fix potential stats race-condition on MT
services
<snip previous discussions>
diff --git a/lib/eal/common/rte_service.c b/lib/eal/common/rte_service.c
index ef31b1f63c..f045e74ef3 100644
--- a/lib/eal/common/rte_service.c
+++ b/lib/eal/common/rte_service.c
@@ -363,9 +363,15 @@ service_runner_do_callback(struct
rte_service_spec_impl *s,
uint64_t start = rte_rdtsc();
s->spec.callback(userdata);
uint64_t end = rte_rdtsc();
- s->cycles_spent += end - start;
+ uint64_t cycles = end - start;
cs->calls_per_service[service_idx]++;
- s->calls++;
+ if (service_mt_safe(s)) {
+ __atomic_fetch_add(&s->cycles_spent, cycles,
__ATOMIC_RELAXED);
+ __atomic_fetch_add(&s->calls, 1,
__ATOMIC_RELAXED);
+ } else {
+ s->cycles_spent += cycles;
+ s->calls++;
This is still a problem from a reader perspective. It is possible that the
writes could be
split while a reader is reading the stats. These need to be atomic adds.
Thanks for pointing out; I do "think" in x86 in terms of load/store tearing;
and on x86
naturally aligned load/stores will not tear. Apologies for missing the ARM
angle here.
I'm not sure how to best encode the difference between tearing & "locked
instructions"
to make things multi-writer safe. But they're not the same thing, and I'd
prefer not pay
the penalty for LOCK instructions (multi-writer) only to satisfy the
non-tearing requirements.
Is there an rte_atomic-* type that is guaranteed as non-tearing?
In that case, changing the type of the calls/cycles_spent variables to such a type to
ensure "non-tearing"
single-reader, single-writer behaviour is enough, instead of forcing
__atomic_fetch_add() everywhere?
Regular read, increment and then atomic store should work without locks
where alignment is correct on most architectures, and doing the store
atomically should prevent any tearing.
This is a good pattern, and provides a noticeable performance benefit
compared to using an atomic add (which is an overkill in single-writer
scenarios), in my experience. The DPDK seqcount implementation
increments the count in this manner.
