jimmystewpot opened a new pull request, #3756:
URL: https://github.com/apache/brpc/pull/3756

   ### What problem does this PR solve?
   
   Issue Number: #3753
   
   Problem Summary:
   
   This issue was discovered while auditing and profiling `brpc` / `bthread` / 
`butil` timer overhead for StarRocks on ARM64 (`__aarch64__`) servers. Even 
though StarRocks' default build configuration uses `WITH_CPU_FREQUENCY=OFF`, 
evaluating `WITH_CPU_FREQUENCY=ON` on ARM64 (where `CNTFRQ_EL0` provides an 
architectural, invariant counter frequency) revealed two defects in 
`src/butil/time.h` and `src/butil/time.cpp`:
   
   1. **Unserialised `cntvct_el0` reads in `butil::detail::clock_cycles()`:**
      On `__aarch64__`, `clock_cycles()` executes `mrs %0, cntvct_el0` without 
an `isb` barrier. Per the ARM Architecture Reference Manual (DDI 0487, Section 
D11.2.2 "Reads of the virtual counter"), reads of `CNTVCT_EL0` can execute 
speculatively and out-of-order relative to earlier loads, stores, or 
conditional branches unless preceded by an `isb`. Without serialisation, short 
timed intervals (`butil::Timer`) can read `cntvct_el0` before an earlier memory 
access or lock acquisition retires.
   2. **Two dependent 64-bit integer divisions on every `cpuwide_time_ns()` 
call:**
      `cpuwide_time_ns()` converts `clock_cycles()` ticks to nanoseconds by 
executing two runtime 64-bit integer divisions by `detail::invariant_cpu_freq` 
(`tsc / cpu_freq` and `remain * 1000000000L / cpu_freq`) on every call, rather 
than precomputing a fixed-point multiplier and shift alongside 
`invariant_cpu_freq` at initialisation time.
   
   ### What is changed and the side effects?
   
   Changed:
   - **`src/butil/time.h` & `src/butil/time.cpp`:**
     - Preceded `mrs %0, cntvct_el0` with `isb` (and a `"memory"` clobber) in 
`butil::detail::clock_cycles()` on `__aarch64__` to prevent out-of-order 
speculative counter hoisting.
     - Added `butil::detail::compute_cpu_frequency_scale()` and 
`butil::detail::cycles_to_ns()` (guarded by `#if defined(__SIZEOF_INT128__)` 
with a 64-bit fallback for 32-bit targets), precomputing 
`invariant_cpu_freq_mult` and `invariant_cpu_freq_shift` inside 
`read_invariant_cpu_frequency()` before publishing `invariant_cpu_freq`.
     - Updated `cpuwide_time_ns()` to convert cycle counts via 128-bit `mul + 
shift` (`cycles_to_ns`), eliminating both runtime 64-bit integer divisions on 
64-bit platforms while preserving exact nanosecond monotonicity and overflow 
safety across decades of uptime.
   - **`test/baidu_time_unittest.cpp`:**
     - Added `BaiduTimeTest.clock_cycles_monotonic_and_serialized` (verifying 
back-to-back monotonicity and cross-thread acquire/release ordering across 
2,000 lock-step handoffs on AArch64).
     - Added `BaiduTimeTest.cpu_frequency_scale_accuracy_and_monotonicity` 
(verifying `<= 1` ns accuracy against exact 128-bit division and strict 
monotonicity across standard ARM64 and x86_64 timer frequencies from 19.2 MHz 
to 3.59 GHz and uptimes up to $10^8$ seconds).
     - Added `BaiduTimeTest.cpuwide_time_with_cpu_frequency` (verifying 
cycle-bracketed `cpuwide_time_ns()` equivalence, `CNTFRQ_EL0` wall-clock 
agreement, and scaling latency).
   
   Side effects:
   - Performance effects:
     - **Scaling Arithmetic Latency (ARM Neoverse-V2, Ubuntu 24.04, GCC 
13.3.0):**
       - Reduces cycle-to-nanosecond scaling math latency from **7.91 ns / op** 
(two 64-bit `udiv` instructions) to **3.04 ns / op** (`mul`/`umulh` + `lsr` — 
**2.6x faster**), and reduces end-to-end serialised `cpuwide_time_ns()` latency 
to **23.71 ns / call** vs **33.36 ns / call** for Linux vDSO 
`monotonic_time_ns()` (**29% faster**).
     - **macOS Apple Silicon (`BaiduTimeTest.cost_of_timer` & 
`cpuwide_time_with_cpu_frequency`, Apple Clang 21):**
       - Reduces `butil::cpuwide_time_ns()` and `butil::Timer::stop()` from 
**18 ns / call** (`clock_gettime`) to **8–9 ns / call** (**2x faster**), and 
reduces pure scaling math from **6 ns / op** (`2x udiv`) to **2 ns / op** (`mul 
+ shift` — **3x faster**).
   
   - Breaking backward compatibility: No. `butil::detail::invariant_cpu_freq` 
(`int64_t`) and all public `butil` timer APIs are preserved with 100% API and 
ABI compatibility.
   
   ---
   ### Check List:
   - Please make sure your changes are compilable.
   - When providing us with a new feature, it is best to add related tests.
   - Please follow [Contributor Covenant Code of 
Conduct](https://github.com/apache/brpc/blob/master/CODE_OF_CONDUCT.md).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to