jimmystewpot opened a new pull request, #3756:
URL: https://github.com/apache/brpc/pull/3756
### What problem does this PR solve?
Issue Number: #3753
Problem Summary:
This issue was discovered while auditing and profiling `brpc` / `bthread` /
`butil` timer overhead for StarRocks on ARM64 (`__aarch64__`) servers. Even
though StarRocks' default build configuration uses `WITH_CPU_FREQUENCY=OFF`,
evaluating `WITH_CPU_FREQUENCY=ON` on ARM64 (where `CNTFRQ_EL0` provides an
architectural, invariant counter frequency) revealed two defects in
`src/butil/time.h` and `src/butil/time.cpp`:
1. **Unserialised `cntvct_el0` reads in `butil::detail::clock_cycles()`:**
On `__aarch64__`, `clock_cycles()` executes `mrs %0, cntvct_el0` without
an `isb` barrier. Per the ARM Architecture Reference Manual (DDI 0487, Section
D11.2.2 "Reads of the virtual counter"), reads of `CNTVCT_EL0` can execute
speculatively and out-of-order relative to earlier loads, stores, or
conditional branches unless preceded by an `isb`. Without serialisation, short
timed intervals (`butil::Timer`) can read `cntvct_el0` before an earlier memory
access or lock acquisition retires.
2. **Two dependent 64-bit integer divisions on every `cpuwide_time_ns()`
call:**
`cpuwide_time_ns()` converts `clock_cycles()` ticks to nanoseconds by
executing two runtime 64-bit integer divisions by `detail::invariant_cpu_freq`
(`tsc / cpu_freq` and `remain * 1000000000L / cpu_freq`) on every call, rather
than precomputing a fixed-point multiplier and shift alongside
`invariant_cpu_freq` at initialisation time.
### What is changed and the side effects?
Changed:
- **`src/butil/time.h` & `src/butil/time.cpp`:**
- Preceded `mrs %0, cntvct_el0` with `isb` (and a `"memory"` clobber) in
`butil::detail::clock_cycles()` on `__aarch64__` to prevent out-of-order
speculative counter hoisting.
- Added `butil::detail::compute_cpu_frequency_scale()` and
`butil::detail::cycles_to_ns()` (guarded by `#if defined(__SIZEOF_INT128__)`
with a 64-bit fallback for 32-bit targets), precomputing
`invariant_cpu_freq_mult` and `invariant_cpu_freq_shift` inside
`read_invariant_cpu_frequency()` before publishing `invariant_cpu_freq`.
- Updated `cpuwide_time_ns()` to convert cycle counts via 128-bit `mul +
shift` (`cycles_to_ns`), eliminating both runtime 64-bit integer divisions on
64-bit platforms while preserving exact nanosecond monotonicity and overflow
safety across decades of uptime.
- **`test/baidu_time_unittest.cpp`:**
- Added `BaiduTimeTest.clock_cycles_monotonic_and_serialized` (verifying
back-to-back monotonicity and cross-thread acquire/release ordering across
2,000 lock-step handoffs on AArch64).
- Added `BaiduTimeTest.cpu_frequency_scale_accuracy_and_monotonicity`
(verifying `<= 1` ns accuracy against exact 128-bit division and strict
monotonicity across standard ARM64 and x86_64 timer frequencies from 19.2 MHz
to 3.59 GHz and uptimes up to $10^8$ seconds).
- Added `BaiduTimeTest.cpuwide_time_with_cpu_frequency` (verifying
cycle-bracketed `cpuwide_time_ns()` equivalence, `CNTFRQ_EL0` wall-clock
agreement, and scaling latency).
Side effects:
- Performance effects:
- **Scaling Arithmetic Latency (ARM Neoverse-V2, Ubuntu 24.04, GCC
13.3.0):**
- Reduces cycle-to-nanosecond scaling math latency from **7.91 ns / op**
(two 64-bit `udiv` instructions) to **3.04 ns / op** (`mul`/`umulh` + `lsr` —
**2.6x faster**), and reduces end-to-end serialised `cpuwide_time_ns()` latency
to **23.71 ns / call** vs **33.36 ns / call** for Linux vDSO
`monotonic_time_ns()` (**29% faster**).
- **macOS Apple Silicon (`BaiduTimeTest.cost_of_timer` &
`cpuwide_time_with_cpu_frequency`, Apple Clang 21):**
- Reduces `butil::cpuwide_time_ns()` and `butil::Timer::stop()` from
**18 ns / call** (`clock_gettime`) to **8–9 ns / call** (**2x faster**), and
reduces pure scaling math from **6 ns / op** (`2x udiv`) to **2 ns / op** (`mul
+ shift` — **3x faster**).
- Breaking backward compatibility: No. `butil::detail::invariant_cpu_freq`
(`int64_t`) and all public `butil` timer APIs are preserved with 100% API and
ABI compatibility.
---
### Check List:
- Please make sure your changes are compilable.
- When providing us with a new feature, it is best to add related tests.
- Please follow [Contributor Covenant Code of
Conduct](https://github.com/apache/brpc/blob/master/CODE_OF_CONDUCT.md).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]