Jeff-Shen-00 opened a new pull request, #70779:
URL: https://github.com/apache/doris/pull/70779
### Background
The Backend is compiled with `-msse4.2` and, by default, `-mavx2`
(`USE_AVX2` in `be/CMakeLists.txt`). Code guarded by `__AVX512F__` /
`__AVX512BW__` / `__AVX512VL__` / `__AVX512DQ__` is therefore never
compiled, even on CPUs that support AVX-512. `build.sh` exposes build
options as environment variables and forwards them to CMake, with
`build.sh --help` as the user-facing documentation; `run-be-ut.sh` does
the same for the unit-test binary.
### What problem does this PR solve?
Issue Number: close #70773
Related PR: None
PR Type: Build/CI/Chore
Problem Summary:
Recent Intel and AMD server CPUs support AVX-512, but there is no way to
build the Backend with AVX-512 enabled, so every AVX-512 code path stays
dead. This adds a `USE_AVX512` option (default OFF) that adds
`-mavx512f -mavx512bw -mavx512vl -mavx512dq` and `-DUSE_AVX512`, and
plumbs it through `build.sh` and `run-be-ut.sh`. Default builds are
unchanged and keep the AVX2 CPU floor.
### Details
- **Affected:** `be/CMakeLists.txt` (option, flags, AVX2 guard), `build.sh`
(help text, default, argument summary, `-DUSE_AVX512`), `run-be-ut.sh`
(default and `-DUSE_AVX512`).
- **Developer and release impact:** `USE_AVX512=ON ./build.sh --be` builds
an AVX-512 binary; the default (`OFF`) produces the same binary as
before. An AVX-512 build requires an AVX-512 capable CPU at run time and
faults on older hosts, which the help text and the option description
state. The UT binary follows the same variable so tests can be built with
the same instruction set as the BE.
- **License and third-party:** no new dependency. The flag set includes DQ
because the flags are inherited by `contrib/faiss`, whose `__AVX512F__`
code paths use DQ intrinsics such as `_mm512_inserti32x8`; without DQ
those files fail with `'__builtin_ia32_inserti32x8' needs target feature
avx512dq,evex512`.
- **Choice of flags:** F for 512-bit integer operations, BW for byte/word
operations, VL for EVEX-encoded 256-bit operations, DQ for the faiss
paths above. `-mavx512f` also defines `__AVX2__`, so `USE_AVX512=ON`
together with `USE_AVX2=OFF` is rejected at configure time instead of
silently producing a binary that assumes AVX2.
- **Rollback:** revert the commit. Default builds are unaffected, and an
AVX-512 build can still be produced by putting the flags in
`EXTRA_CXX_FLAGS`.
### Testing
- Full ASAN BE build with this flag set: 14006/14006 targets, exit 0.
Without DQ it failed in `contrib/faiss`, which is why DQ is in the set.
- `cmake` configure, default: `-- USE_AVX512 is OFF`; 0 of 1258 Doris
source TUs in `compile_commands.json` carry `-mavx512*`.
- `cmake` configure, `-DUSE_AVX512=ON`: 1258 of 1258 Doris source TUs carry
`-mavx512f -mavx512bw -mavx512vl -mavx512dq` and `-DUSE_AVX512`.
- `cmake` configure, `-DUSE_AVX512=ON -DUSE_AVX2=OFF`: fails with
`USE_AVX512 requires USE_AVX2=ON: -mavx512f implies AVX2.`
(`be/CMakeLists.txt:402`), as intended.
- `bash -n build.sh run-be-ut.sh` passes.
- The branch was rebased onto master `635fab23d235` before submission; the
change itself is identical after the rebase, and the configure checks
above were run against it on the branch's original base. This checkout
cannot configure current master end to end because its
`thirdparty/installed` predates mongo-c-driver (no `bson-2.0` /
`mongoc-2.0`), which is unrelated to this change.
### Notes for Reviewers
- The flags are added in the existing `if (... x86 OR x86_64)` SIMD block,
next to `USE_AVX2` and before the `add_subdirectory` calls, so every
Doris target and every contrib library built from `be/CMakeLists.txt`
inherits them.
- `USE_AVX512` is a `BOOL` cache variable, so a manual
`cmake -DUSE_AVX512=ON` works without going through `build.sh`.
### Release note
New build option `USE_AVX512` (default OFF) to build the Backend with
AVX-512 instructions (F/BW/VL/DQ). The resulting binary requires an
AVX-512 capable CPU.
### Check List (For Author)
- Behavior changed: No by default; Yes only with `USE_AVX512=ON` (see
Details)
- Does this need documentation: No, `build.sh --help` documents the option.
- Pre-submission status: Partially verified: configure-level checks and
`bash -n` were re-run; the full BE build was done on the branch base, not
after the rebase.
### AI Disclosure
- Tool: Claude Code
- Generated: the original branch commit (which carries a Claude Code
co-author), the rebase onto master, and this description.
- Human review: pending; the author has not yet reviewed the generated
parts.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]