[
https://issues.apache.org/jira/browse/IMPALA-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18118198#comment-18118198
]
ASF subversion and git services commented on IMPALA-15369:
----------------------------------------------------------
Commit 74159a83430aa0b9ea0ac4dcb9a4c965ebf3a681 in impala's branch
refs/heads/master from Zoltan Borok-Nagy
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=74159a834 ]
IMPALA-15369: Check for the CRC32 extension at startup on ARM64
On aarch64, Impala is built with -march=armv8-a+crc, and native code
and codegen use CRC32 instructions for hashing (HashUtil::Hash(),
LlvmCodeGen::GetHashFunction()) without checking for them at runtime.
The CRC32 extension is optional in ARMv8.0, so on such CPUs the daemons
crash with SIGILL once a CRC32 code path runs, instead of failing at
startup with a clear message.
This change adds a CpuInfo::CRC32 hardware flag. CpuInfo::Init() now
also parses the 'Features' line of /proc/cpuinfo, which is where the
kernel lists the CPU features on aarch64 (x86_64 uses 'flags').
CpuInfo::EnforceCpuRequirements() fails on aarch64 if CRC32 is not
supported. All daemons already call it from InitCommonRuntime() and
exit with an error message if it fails, like for the AVX/AVX2
requirement on x86_64.
Testing:
- Core tests show no new errors on ARM machine
Change-Id: Ic01f8f8009e6e2d8dbfd324d83353b0d4dbf1d4c
Assisted-by: Claude Opus 5.5 (Claude Code)
Reviewed-on: http://gerrit.cloudera.org:8080/24888
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Check for the CRC32 extension at startup on ARM64
> -------------------------------------------------
>
> Key: IMPALA-15369
> URL: https://issues.apache.org/jira/browse/IMPALA-15369
> Project: IMPALA
> Issue Type: Improvement
> Components: Backend
> Reporter: Zoltán Borók-Nagy
> Assignee: Zoltán Borók-Nagy
> Priority: Major
>
> On x86-64, {{CpuInfo::EnforceCpuRequirements()}} makes every Impala daemon
> (impalad, catalogd, statestored, admissiond) exit with a clear error message
> when the CPU lacks AVX2, or AVX in legacy mode (IMPALA-9690). There is no
> such check on
> ARM64: the function does nothing outside {{{}#ifdef _{_}x86_64{_}_{}}}.
> The ARM64 build still assumes the CRC32 extension. Native code and
> cross-compiled IR are compiled with {{-march=armv8-a+crc}}
> (be/CMakeLists.txt). CRC32 instructions are used without checking for the
> extension:
> * {{SSE4_crc32_u8/u16/u32/u64}} in be/src/util/sse-util.h emit
> {{crc32cb/ch/cw/cx}} as inline asm, or ACLE {{__crc32c*}} intrinsics in IR.
> * {{HashUtil::Hash()}} always selects {{CrcHash()}} on aarch64
> ({{{}IS_AARCH64 || ...{}}}).
> * {{LlvmCodeGen::GetHashFunction()}} emits {{aarch64_crc32c*}} intrinsics,
> also skipping the feature check on aarch64.
> CRC32 is optional in ARMv8.0 and only mandatory from ARMv8.1. On an ARMv8.0
> CPU without it, Impala does not fail at startup. It crashes with SIGILL once
> one of the code paths above runs: join/aggregation hash
> tables, Parquet dictionary encoding, the scheduler's hash ring, etc.
> Such CPUs are rare today, but the failure is confusing and hard to diagnose.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
