jimmystewpot opened a new pull request, #3754:
URL: https://github.com/apache/brpc/pull/3754

   ### What problem does this PR solve?
   
   Issue Number: #3753
   
   Problem Summary:
   
   This issue was discovered while profiling StarRocks (which relies heavily on 
`brpc` / `bthread` / `butil` for RPC and data framing) on ARM64 (`__aarch64__`) 
servers.
   
   `src/butil/crc32c.cc` currently accelerates CRC32C on x86_64 (`__SSE4_2__`) 
and RISC-V (`__riscv_zbc` / `__riscv_zvbc`), but has **no ARM64 (`__aarch64__`) 
hardware path**. On ARM64:
   - `butil::crc32c::IsFastCrc32Supported()` returns `false`.
   - `butil::crc32c::Extend()` falls back to `ExtendImpl<SlowCRC32Functor>`, a 
4-byte-at-a-time software lookup table (~1.1–1.2 GiB/s), despite hardware 
`FEAT_CRC32` and `FEAT_PMULL` being standard across modern ARM64 server and 
desktop CPUs (ARM Neoverse N1/N2/V1/V2, AWS Graviton 2/3/4, Ampere Altra, and 
Apple Silicon).
   
   ### What is changed and the side effects?
   
   Changed:
   - **`src/butil/crc32c.cc`:**
     - Added an `#if defined(__aarch64__)` hardware-accelerated CRC32C path 
using `<arm_acle.h>` (`__crc32cb`, `__crc32ch`, `__crc32cw`, `__crc32cd`) and 
`<arm_neon.h>` (`vmull_p64`).
     - Annotated the hardware routines with per-function compiler target 
attributes (`__attribute__((target("+crc")))` / 
`__attribute__((target("+crc+crypto")))` on GCC, and 
`__attribute__((target("crc")))` / `__attribute__((target("crc,aes")))` on 
Clang) so that default baseline `-march=armv8-a` builds compile cleanly with 
zero build-flag changes required.
     - Added cached runtime CPU capability detection (`isArmCrc32()` and 
`isArmPmull()`) using `getauxval(AT_HWCAP)` (`HWCAP_CRC32` and `HWCAP_PMULL`) 
on Linux and `sysctlbyname` on macOS, wired into `Choose_Extend()` and 
`IsFastCrc32Supported()` so a single baseline binary automatically selects the 
fastest supported path (`CRC32+PMULL`, scalar `CRC32`, or software table 
fallback) at runtime.
     - Implemented 3-way pipelined `__crc32cd` lanes combined via `vmull_p64` 
carry-less multiplication across `1008`-byte (`3 x 336B`) and `192`-byte (`3 x 
64B`) blocks when `FEAT_PMULL` is available, with scalar 
`__crc32cd`/`__crc32cw`/`__crc32ch`/`__crc32cb` handling short buffers, 
unaligned prologues, and tails.
   - **`test/crc32c_unittest.cc`:**
     - Added unit tests verifying `IsFastCrc32Supported()` on AArch64, 
bit-for-bit agreement against an independent bitwise Castagnoli reference 
across alignments `0..15`, all tail lengths `0..512`, stripe boundaries 
(`191..3073`), large buffers (`1 KiB..64 KiB`), and multi-chunk incremental 
`Extend()` calls, as well as a throughput benchmark.
   
   Side effects:
   - Performance effects:
     - **Linux AArch64 Benchmark (ARM Neoverse-V2, Ubuntu 24.04, GCC 13.3.0, 
compiled with baseline `-march=armv8-a`):**
       | Buffer Size | Before (`SlowCRC32Functor` software table) | After 
(Runtime-detected `+crc+crypto`) | Speedup |
       | :--- | :---: | :---: | :---: |
       | **64 B** | 1,189.4 MiB/s (1.16 GiB/s) | **13,015.0 MiB/s** (12.71 
GiB/s) | **10.9x** |
       | **256 B** | 1,215.0 MiB/s (1.19 GiB/s) | **16,368.7 MiB/s** (15.99 
GiB/s) | **13.5x** |
       | **1 KiB (1024 B)** | 1,224.1 MiB/s (1.20 GiB/s) | **20,323.6 MiB/s** 
(19.85 GiB/s) | **16.6x** |
       | **4 KiB (4096 B)** | 1,226.1 MiB/s (1.20 GiB/s) | **20,983.9 MiB/s** 
(20.49 GiB/s) | **17.1x** |
       | **64 KiB (65536 B)** | 1,225.6 MiB/s (1.20 GiB/s) | **21,232.5 MiB/s** 
(20.73 GiB/s) | **17.3x** |
     - **macOS Apple Silicon Benchmark (Apple Clang 21):**
       - Increases 64 KiB CRC32C throughput from **1,147 MiB/s** (1.12 GiB/s) 
to **28,881 MiB/s** (28.20 GiB/s — **25.2x faster**).
   
   - Breaking backward compatibility: No. Preserves 100% API, ABI, and 
wire-format compatibility (`CRC.*`, `brpc_checksum_unittest`, and 
`RecordIOTest.*` all pass).
   
   ---
   ### Check List:
   - Please make sure your changes are compilable.
   - When providing us with a new feature, it is best to add related tests.
   - Please follow [Contributor Covenant Code of 
Conduct](https://github.com/apache/brpc/blob/master/CODE_OF_CONDUCT.md).


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to