This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new c298259d425 [chore](build) Build multi-architecture all-in-one images
by default (#70490)
c298259d425 is described below
commit c298259d425b0418568bc8f25bbb397dd34b1e16
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Tue Nov 17 11:30:18 2026 +0800
[chore](build) Build multi-architecture all-in-one images by default
(#70490)
### Background
The Doris 4.1 all-in-one build script assembles base and full images
from the
FE, BE and meta-service component images. It supports an OCI image index
with
amd64 and arm64 variants and permits an explicit single-platform
override.
### What problem does this PR solve?
Issue Number: None
Related PR: None
PR Type: Build/CI/Chore
Problem Summary:
Without --platform, the script previously built only the Docker host's
architecture. Default to linux/amd64,linux/arm64 so a normal
component-image
build produces both variants. Update the help and README examples and
reject
multi-platform builds from a directory containing single-architecture
binaries.
The smoke test accepts both paimon-scanner and paimon-connector
directory names
so it validates the Paimon payload in both 4.1.3 and 4.1.5 packages.
### Details
- **Affected:** docker/runtime/all-in-one/4.1/build.sh,
resource/smoke-test.sh
and the README.
- **Developer and release impact:** The default builds both
architectures for
both flavors. Single-platform builds still use --platform. Local output,
release tarballs and --test require a matching single platform.
Multi-platform
local builds continue to require the containerd image store; --push
retains
registry output support.
- **License and third-party:** None.
- **Rollback:** Restore the previous platform default and associated
examples.
### Testing
- bash -n and ShellCheck passed for the build script; git diff --check
passed.
- Eleven offline checks with a temporary Docker substitute passed:
default
platforms for both flavor tags, explicit amd64 and arm64 overrides, push
without containerd, rejection of the classic store and multi-platform
smoke
tests, rejection of multi-platform local output and tarballs,
single-platform
directory builds, and the help text.
- Built 4.1.5 base and full images for linux/amd64 and linux/arm64 using
the
default platform selection. Verified both platforms, version/flavor
labels
and the packaged smoke-test checksum.
- resource/smoke-test.sh passed on native arm64 for 4.1.3 base, 4.1.5
base and
4.1.5 full: JNI payload, flavor pruning, insert/select, Stream Load,
aggregation and schema changes. The 4.1.5 FE and BE report
doris-4.1.5-release-917b20be838.
- The original Paimon check failed on 4.1.5; the updated check accepts
the
release's paimon-connector directory and the legacy paimon-scanner
directory.
- bash -n and ShellCheck at warning level passed for the updated smoke
test.
- amd64 runtime execution was not tested on this arm64 host.
---
docker/runtime/all-in-one/4.1/README.md | 26 +++++++++++--------
docker/runtime/all-in-one/4.1/build.sh | 30 ++++++++++++++--------
.../runtime/all-in-one/4.1/resource/smoke-test.sh | 7 ++++-
3 files changed, 41 insertions(+), 22 deletions(-)
diff --git a/docker/runtime/all-in-one/4.1/README.md
b/docker/runtime/all-in-one/4.1/README.md
index b18e0fb9192..9b6e1f5f7b1 100644
--- a/docker/runtime/all-in-one/4.1/README.md
+++ b/docker/runtime/all-in-one/4.1/README.md
@@ -50,25 +50,31 @@ is enabled, because it adds the compressed blobs to the
unpacked snapshot.
## Build
-A plain run builds for the host architecture only. Multi-arch needs an explicit
-`--platform`; the header the script prints says which it is doing.
+A plain run builds both `linux/amd64` and `linux/arm64`. Pass `--platform` with
+one platform for a single-architecture build; the script prints the selected
+platforms before building. Keeping both platforms locally requires the
+containerd image store; otherwise pass `--push`.
```shell
-# both tags, from the official 4.1.3 component images, host architecture only
+# both tags, from the official 4.1.3 component images, amd64 + arm64
./build.sh -v 4.1.3
-# base tag only, then smoke test it
-./build.sh -v 4.1.3 -f base -t
+# base tag for arm64, then smoke test it (use linux/amd64 on x86)
+./build.sh -v 4.1.3 -f base --platform linux/arm64 -t
-# from a locally built ./output instead (build.sh --fe --be --cloud; without
+# from an arm64 ./output instead (build.sh --fe --be --cloud; without
# --cloud there is no ms/ and the image cannot serve the cloud compose file)
-./build.sh -v dev -s local
+./build.sh -v dev -s local --platform linux/arm64
# from an extracted release tarball, e.g. while the component images for a
# new release are not on Docker Hub yet
-./build.sh -v 4.1.4 -s tarball --tarball-dir ~/apache-doris-4.1.4-bin-arm64
+./build.sh -v 4.1.4 -s tarball --platform linux/arm64 --tarball-dir
~/apache-doris-4.1.4-bin-arm64
```
+`--test` requires a single platform that runs on the test host. Local output
+and release tarballs contain binaries for one architecture, so `--source local`
+and `--source tarball` require a single `--platform` matching those binaries.
+
`./build.sh --help` lists the rest. The build context is the repository root,
narrowed to a few KB by `Dockerfile.dockerignore`; a local `./output` or a
tarball directory is passed as BuildKit named contexts instead, so it can live
@@ -94,7 +100,7 @@ its own, so no per-arch directories have to be staged by
hand.
is foreign to the machine:
```shell
-./build.sh -v 4.1.3 --platform linux/amd64,linux/arm64 --push
+./build.sh -v 4.1.3 --push
```
Keeping the result locally instead of pushing works only with the containerd
@@ -185,7 +191,7 @@ named as a string literal under `be/src/format_v2/jni/` and
| class | directory | tag |
|---|---|---|
-| `PaimonJniScanner` | `paimon-scanner` | both |
+| `PaimonJniScanner` | `paimon-scanner` / `paimon-connector` (depends on the
4.1.x release) | both |
| `IcebergSysTableJniScanner` | `iceberg-metadata-scanner` | both |
| `JdbcJniScanner` | `jdbc-scanner` | both |
| (`JniWriter`) | `java-writer` | both |
diff --git a/docker/runtime/all-in-one/4.1/build.sh
b/docker/runtime/all-in-one/4.1/build.sh
index c06eefb6f87..21d7537ce10 100755
--- a/docker/runtime/all-in-one/4.1/build.sh
+++ b/docker/runtime/all-in-one/4.1/build.sh
@@ -29,7 +29,7 @@ VERSION=""
SOURCE="image"
FLAVORS=""
STRIP_MODE="debug"
-PLATFORM=""
+PLATFORM="linux/amd64,linux/arm64"
LOCAL_OUTPUT="output"
TARBALL_DIR=""
RUN_TEST=false
@@ -64,8 +64,10 @@ Options:
debug -> strip --strip-debug, keeps .symtab
full -> strip -s
none -> ship doris_be and doris_cloud as-is (+2.2
GB)
- --platform <p> Target platform(s), e.g. linux/amd64 or
- linux/amd64,linux/arm64. Comma-separated values produce
+ --platform <p> Target platform(s) (default: linux/amd64,linux/arm64).
+ Pass one platform for a single-architecture build.
+ Local output and tarballs need one matching platform.
+ Comma-separated values produce
one multi-arch tag (an OCI image index) that resolves
to
the right image per host, the way apache/doris:fe-*
does.
Keeping a multi-platform result locally needs the
@@ -74,16 +76,19 @@ Options:
-i, --image <name> Image name without tag (default: apache/doris)
--no-cache Pass --no-cache to docker build.
-t, --test Run resource/smoke-test.sh against each image built.
+ Requires a single --platform that runs on this host.
-h, --help This message.
Examples:
- ./build.sh -v 4.1.3 # both flavors from the official
images
- ./build.sh -v 4.1.3 -f base -t # base only, then smoke test it
- ./build.sh -v dev -s local -f full # from a local ./output (build.sh
--fe --be --cloud)
- ./build.sh -v 4.1.4 -s tarball --tarball-dir ~/apache-doris-4.1.4-bin-arm64
+ ./build.sh -v 4.1.3 # both flavors, amd64 + arm64
+ ./build.sh -v 4.1.3 -f base --platform linux/arm64 -t
+ # base for arm64, then smoke test it
+ ./build.sh -v dev -s local -f full --platform linux/arm64
+ # from an arm64 ./output (build.sh
--fe --be --cloud)
+ ./build.sh -v 4.1.4 -s tarball --platform linux/arm64 --tarball-dir
~/apache-doris-4.1.4-bin-arm64
# from a release tarball, e.g.
before the
# component images are on Docker Hub
- ./build.sh -v 4.1.3 --platform linux/amd64,linux/arm64 --push
+ ./build.sh -v 4.1.3 --push
# one multi-arch tag for both
Building a foreign architecture goes through emulation, and the two apt layers
@@ -169,7 +174,11 @@ command -v docker >/dev/null || { echo "error: docker not
found" >&2; exit 1; }
# nowhere to put a second architecture, so the build has to go straight to a
# registry.
MULTI_PLATFORM=false
-case "${PLATFORM}" in *,*) MULTI_PLATFORM=true ;; esac
+case "${PLATFORM}" in *,*) MULTI_PLATFORM=true ;; *) ;; esac
+if [[ "${MULTI_PLATFORM}" == true && -n "${ARTIFACT_DIR}" ]]; then
+ echo "error: --source ${SOURCE} contains single-architecture binaries;
pass --platform with the matching single platform" >&2
+ exit 1
+fi
if [[ "${MULTI_PLATFORM}" == true && "${PUSH}" != true ]]; then
if ! docker info --format '{{range .DriverStatus}}{{.}}{{end}}'
2>/dev/null | grep -q containerd; then
echo "error: a multi-platform build needs --push, or the containerd
image store" >&2
@@ -190,8 +199,7 @@ if ! docker buildx version >/dev/null 2>&1; then
fi
fi
-# Say which platforms are being built. Leaving --platform unset means the host
-# architecture only, which is easy to mistake for a multi-arch build.
+# Report the selected platforms, including the multi-architecture default.
if [[ -n "${PLATFORM}" ]]; then
platform_note="${PLATFORM}"
else
diff --git a/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
b/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
index 1109db5d0fb..a87ac867ce4 100755
--- a/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
+++ b/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
@@ -90,8 +90,13 @@ for d in be/lib/java_extensions/preload-extensions \
done
step "flavor payload matches ${FLAVOR}"
+# Paimon's package directory differs across 4.1.x releases.
+in_image "be/lib/java_extensions/paimon-scanner" \
+ || in_image "be/lib/java_extensions/paimon-connector" \
+ || fail "missing from base payload: paimon-scanner or paimon-connector"
+echo " ok Paimon scanner"
# Present in both tags.
-for d in paimon-scanner iceberg-metadata-scanner jdbc-scanner java-writer; do
+for d in iceberg-metadata-scanner jdbc-scanner java-writer; do
in_image "be/lib/java_extensions/${d}" || fail "missing from base payload:
${d}"
echo " ok ${d}"
done
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]