This is an automated email from the ASF dual-hosted git repository.
joemcdonnell pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/impala.git
The following commit(s) were added to refs/heads/master by this push:
new 89d2b2350 IMPALA-14139: Enable Impala builds on Ubuntu 24.04
89d2b2350 is described below
commit 89d2b235096bd01fd779ea8bcbb41c49c4688144
Author: Laszlo Gaal <[email protected]>
AuthorDate: Fri Aug 15 14:06:06 2025 +0200
IMPALA-14139: Enable Impala builds on Ubuntu 24.04
Update the following elements of the Impala build environment to enable
builds on Ubuntu 24.04:
- Recognize and handle (where necessary) Ubuntu 24.04 in various
bootstrap scripts (bootstrap_system.sh, bootstrap_toolchain.py, etc.)
- Bump IMPALA_TOOLCHAIN_ID to an official toolchain build that contains
Ubuntu 24.04-specific binary packages
- Bump binutils to 2.42, and
- Bump the GDB version to 12.1-p1, as required by the new toolchain
version
- Update unique_ptr usage syntax in be/src/util/webserver-test.cc to
compensate for new GLIBC funtion prototypes:
System headers in Ubuntu 24.04 adopted attributes on several widely
used function prototypes. Such attributes are not considered to be part
of the function's signature during template evaluation, so GCC throws a
warning when such a function is passed as a template argument, which
breaks the build, as warnings are treated as errors.
webserver-test.cc uses pclose() as the deleter for a unique_ptr in a
utility function. This patch encapsulates pclose() and its attributes in
an explicit specialization for std::default_delete<>, "hiding" the
attributes inside a functor.
The particular solution was inspired by Anton-V-K's proposal in
https://gist.github.com/t-mat/5849549
This commit builds on an earlier patch for the same purpose by Michael
Smith: https://gerrit.cloudera.org/c/23058/
Change-Id: Ia4454b0c359dbf579e6ba2f9f9c44cfa3f1de0d2
Reviewed-on: http://gerrit.cloudera.org:8080/23384
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Michael Smith <[email protected]>
Reviewed-by: Joe McDonnell <[email protected]>
---
be/src/util/webserver-test.cc | 10 +++++++++-
bin/bootstrap_build.sh | 2 +-
bin/bootstrap_system.sh | 27 +++++++++++++++++++++------
bin/bootstrap_toolchain.py | 3 ++-
bin/impala-config.sh | 12 ++++++------
5 files changed, 39 insertions(+), 15 deletions(-)
diff --git a/be/src/util/webserver-test.cc b/be/src/util/webserver-test.cc
index a3e83c2ee..c9c308146 100644
--- a/be/src/util/webserver-test.cc
+++ b/be/src/util/webserver-test.cc
@@ -66,10 +66,18 @@ const string TO_ESCAPE_KEY = "ToEscape";
const string TO_ESCAPE_VALUE = "<script language='javascript'>";
const string ESCAPED_VALUE = "<script language='javascript'>";
+// Specialize a deleter for the pipe in exec() below
+template <>
+struct std::default_delete<FILE> {
+ void operator()(FILE* p) const {
+ pclose(p);
+ }
+};
+
string exec(const char* cmd) {
std::array<char, 1024> buffer;
string result;
- unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
+ unique_ptr<FILE> pipe(popen(cmd, "r"));
if (!pipe) {
throw std::runtime_error(Substitute("popen() failed with $0", errno));
}
diff --git a/bin/bootstrap_build.sh b/bin/bootstrap_build.sh
index 56b49ae20..38b3b8d0a 100755
--- a/bin/bootstrap_build.sh
+++ b/bin/bootstrap_build.sh
@@ -18,7 +18,7 @@
# under the License.
# This script builds Impala from scratch. It is known to work on Ubuntu
versions 16.04,
-# 18.04, 20.04, and 22.04. To run it you need to have:
+# 18.04, 20.04, 22.04 and 24.04. To run it you need to have:
#
# 1. At least 8GB of free disk space
# 4. A connection to the internet (parts of the build download dependencies)
diff --git a/bin/bootstrap_system.sh b/bin/bootstrap_system.sh
index cf3cea83a..b91df6fc0 100755
--- a/bin/bootstrap_system.sh
+++ b/bin/bootstrap_system.sh
@@ -78,6 +78,7 @@ UBUNTU16=
UBUNTU18=
UBUNTU20=
UBUNTU22=
+UBUNTU24=
IN_DOCKER=
if [[ -f /etc/redhat-release ]]; then
REDHAT=true
@@ -119,8 +120,12 @@ else
then
UBUNTU22=true
echo "Identified Ubuntu 22.04 system."
+ elif [[ $DISTRIB_RELEASE = 24.04 ]]
+ then
+ UBUNTU24=true
+ echo "Identified Ubuntu 24.04 system."
else
- echo "This script supports Ubuntu versions 16.04, 18.04, 20.04, or
22.04" >&2
+ echo "This script supports Ubuntu versions 16.04, 18.04, 20.04, 22.04,
or 24.04" >&2
exit 1
fi
else
@@ -166,6 +171,12 @@ function ubuntu22 {
fi
}
+function ubuntu24 {
+ if [[ "$UBUNTU24" == true ]]; then
+ "$@"
+ fi
+}
+
# Helper function to execute following command only on RedHat
function redhat {
if [[ "$REDHAT" == true ]]; then
@@ -262,9 +273,12 @@ if [[ $UBUNTU20 == true ]]; then
fi
fi
-# Required by Kudu in the minicluster
-ubuntu20 apt-get --yes install libtinfo5
-ubuntu22 apt-get --yes install libtinfo5
+# Required by Kudu in the minicluster. Older Kudu versions depend on libtinfo5,
+# versions that can be compiled for Ubuntu 24.04 depend on libtinfo6.
+ubuntu20 apt-get --yes install libtinfo5 libtinfo6
+ubuntu22 apt-get --yes install libtinfo5 libtinfo6
+ubuntu24 apt-get --yes install libtinfo6
+
ARCH_NAME=$(uname -p)
if [[ $ARCH_NAME == 'aarch64' ]]; then
ubuntu apt-get --yes install unzip pkg-config flex maven python3-pip
build-essential \
@@ -328,14 +342,14 @@ function setup_python2() {
redhat8 setup_python2
redhat8 pip install --user argparse
-# Point Python to Python 3 for Redhat 9 and Ubuntu 22
+# Point Python to Python 3 for Redhat 9 and Ubuntu 22, or newer
function setup_python3() {
# If python is already set, then use it. Otherwise, try to point python to
python3.
if ! command -v python > /dev/null; then
if command -v python3 ; then
# Newer OSes (e.g. Redhat 9 and equivalents) make it harder to get
Python 2, and we
# need to start using Python 3 by default.
- # For these new OSes (Ubuntu 22, Redhat 9), there is no alternative
entry for
+ # For these new OSes (Ubuntu 22+, Redhat 9), there is no alternative
entry for
# python, so we need to create one from scratch.
if command -v alternatives > /dev/null; then
if sudo alternatives --list | grep python > /dev/null ; then
@@ -359,6 +373,7 @@ function setup_python3() {
redhat9 setup_python3
ubuntu22 setup_python3
+ubuntu24 setup_python3
# CentOS repos don't contain ccache, so install from EPEL
redhat sudo yum install -y epel-release
diff --git a/bin/bootstrap_toolchain.py b/bin/bootstrap_toolchain.py
index b3dda414b..20e759c26 100755
--- a/bin/bootstrap_toolchain.py
+++ b/bin/bootstrap_toolchain.py
@@ -88,7 +88,8 @@ OS_MAPPING = [
OsMapping('ubuntu16', "ec2-package-ubuntu-16-04"),
OsMapping('ubuntu18', "ec2-package-ubuntu-18-04"),
OsMapping('ubuntu20', "ec2-package-ubuntu-20-04"),
- OsMapping('ubuntu22', "ec2-package-ubuntu-22-04")
+ OsMapping('ubuntu22', "ec2-package-ubuntu-22-04"),
+ OsMapping('ubuntu24', "ec2-package-ubuntu-24-04")
]
diff --git a/bin/impala-config.sh b/bin/impala-config.sh
index 7cf9b360a..c0ab7ca3c 100755
--- a/bin/impala-config.sh
+++ b/bin/impala-config.sh
@@ -81,13 +81,13 @@ export USE_AVRO_CPP=${USE_AVRO_CPP:=false}
# moving to a different build of the toolchain, e.g. when a version is bumped
or a
# compile option is changed. The build id can be found in the output of the
toolchain
# build jobs, it is constructed from the build number and toolchain git hash
prefix.
-export IMPALA_TOOLCHAIN_BUILD_ID_AARCH64=108-a38e3142e7
-export IMPALA_TOOLCHAIN_BUILD_ID_X86_64=541-a38e3142e7
+export IMPALA_TOOLCHAIN_BUILD_ID_AARCH64=126-264503de7e
+export IMPALA_TOOLCHAIN_BUILD_ID_X86_64=568-264503de7e
export IMPALA_TOOLCHAIN_REPO=\
${IMPALA_TOOLCHAIN_REPO:-https://github.com/cloudera/native-toolchain.git}
export IMPALA_TOOLCHAIN_BRANCH=${IMPALA_TOOLCHAIN_BRANCH:-master}
export IMPALA_TOOLCHAIN_COMMIT_HASH=\
-${IMPALA_TOOLCHAIN_COMMIT_HASH-a38e3142e70ce74cdf18c3527ece27835adaa58f}
+${IMPALA_TOOLCHAIN_COMMIT_HASH-264503de7ee132bba093c6cd0f1d309c2e10ca94}
# Compare the build ref in build IDs by removing everything
up-to-and-including the
# first hyphen.
if [ "${IMPALA_TOOLCHAIN_BUILD_ID_AARCH64#*-}" \
@@ -115,7 +115,7 @@ else
export IMPALA_AVRO_VERSION=1.7.4-p5
fi
unset IMPALA_AVRO_URL
-export IMPALA_BINUTILS_VERSION=2.35.1
+export IMPALA_BINUTILS_VERSION=2.42
unset IMPALA_BINUTILS_URL
export IMPALA_BOOST_VERSION=1.74.0-p1
unset IMPALA_BOOST_URL
@@ -137,7 +137,7 @@ export IMPALA_FLATBUFFERS_VERSION=1.9.0-p1
unset IMPALA_FLATBUFFERS_URL
export IMPALA_GCC_VERSION=10.4.0
unset IMPALA_GCC_URL
-export IMPALA_GDB_VERSION=12.1
+export IMPALA_GDB_VERSION=12.1-p1
unset IMPALA_GDB_URL
export IMPALA_GFLAGS_VERSION=2.2.0-p2
unset IMPALA_GFLAGS_URL
@@ -1118,7 +1118,7 @@ fi
# overall build type) and does not apply when using a local Kudu build.
export USE_KUDU_DEBUG_BUILD=${USE_KUDU_DEBUG_BUILD-false}
-export IMPALA_KUDU_VERSION=${IMPALA_KUDU_VERSION-"e742f86f6d"}
+export IMPALA_KUDU_VERSION=${IMPALA_KUDU_VERSION-"54f3bd31c"}
export
IMPALA_KUDU_HOME=${IMPALA_TOOLCHAIN_PACKAGES_HOME}/kudu-$IMPALA_KUDU_VERSION
export IMPALA_KUDU_JAVA_HOME=\
${IMPALA_TOOLCHAIN_PACKAGES_HOME}/kudu-${IMPALA_KUDU_VERSION}/java
