Laszlo Gaal has uploaded this change for review. ( http://gerrit.cloudera.org:8080/22583
Change subject: IMPALA-13825: Extend Docker container build to custom base images ...................................................................... IMPALA-13825: Extend Docker container build to custom base images Downstream system vendors, users and customers have lately expressed interest in consuming Impala in containerized forms, taking advantage of various specialized, hardened container base image offerings, like container offerings based on the Wolfi project by Chainguard; see: https://github.com/wolfi-dev. This patch enables Impala container images to be built on top of custom base images, and adds an implementation example that uses the publicly available Wolfi base image. Building a customized Docker image follows a hybrid approach. Instead of replicating the complete Impala build process inside a Wolfi container for a fully native binary build, it relies on an existing build platform that is compatible with the binary packages available inside the custom container image. For Wolfi the Impala binaries are supplied by the Red Hat 9 build of Impala. This is made possible by the fact that major library dependencies of Impala have the same versions on Wolfi OS and Red Hat 9, so binaries built on Red Hat 9 can be run on Wolfi with no changes. The binaries produced by the regular build process are then installed into a Docker image built on top of an explicitly specified custom base image. The selection of a custom base image is controlled by two environment variables: - USE_HARDENED_IMPALA_BASE_IMAGE: If set to 'true', triggers the use of the custom image. When set to 'false' or left unspecified, the Docker base image is selected by the existing logic of matching the build platform's operating system. - IMPALA_HARDENED_DOCKER_BASE: specifies the URI of the base image These environment variables can be overridden from the environment. They are reported at the end of bin/impala-config.sh where important environment variables are listed. They are also added to the list of variables explicitly preserved during Dockerised minicluster testing to ensure that they work in the context of Jenkins jobs running the dockerised tests. The unified script that installs Impala's required dependencies into the container image is extended for Wolfi (or other environments that use APK packages). A new script is added to install Bash in the Docker image if it is missing. Impala build scripts (including the scripts used during Docker image builds) as well as container startup scripts require Bash, but minimal container base images usually omit it, favoring a smaller alternative. To improve the debugging experience for a containerized Impala minicluster, the minicluster starter script bin/start-impala-cluster.py is extended with the following features: - mounts /etc/localtime from the host into each launched container to ensure that the host and all containers share the same local time and timezone. - sets up a directory for each container to collect the Java VMs error files (hs_err_pidNNNN.log) from the containers. - adds the --mount_sources command line parameter, which mounts the complete $IMPALA_HOME subtree into the container at /opt/impala/sources to make source code available inside the container for easier debugging. Change-Id: Ia5e39f399664fe66f3774caa316ed5d4df24befc --- M bin/impala-config.sh M bin/jenkins/dockerized-impala-bootstrap-and-test.sh M bin/start-impala-cluster.py M docker/CMakeLists.txt M docker/daemon_entrypoint.sh M docker/docker-build.sh M docker/impala_base/Dockerfile M docker/impala_profile_tool/Dockerfile A docker/install_bash_if_needed.sh M docker/install_os_packages.sh M docker/setup_build_context.py 11 files changed, 219 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/22583/1 -- To view, visit http://gerrit.cloudera.org:8080/22583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia5e39f399664fe66f3774caa316ed5d4df24befc Gerrit-Change-Number: 22583 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal <[email protected]>
