This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/master by this push:
     new c298259d425 [chore](build) Build multi-architecture all-in-one images 
by default (#70490)
c298259d425 is described below

commit c298259d425b0418568bc8f25bbb397dd34b1e16
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Tue Nov 17 11:30:18 2026 +0800

    [chore](build) Build multi-architecture all-in-one images by default 
(#70490)
    
    ### Background
    
    The Doris 4.1 all-in-one build script assembles base and full images
    from the
    FE, BE and meta-service component images. It supports an OCI image index
    with
    amd64 and arm64 variants and permits an explicit single-platform
    override.
    
    ### What problem does this PR solve?
    
    Issue Number: None
    Related PR: None
    PR Type: Build/CI/Chore
    
    Problem Summary:
    
    Without --platform, the script previously built only the Docker host's
    architecture. Default to linux/amd64,linux/arm64 so a normal
    component-image
    build produces both variants. Update the help and README examples and
    reject
    multi-platform builds from a directory containing single-architecture
    binaries.
    
    The smoke test accepts both paimon-scanner and paimon-connector
    directory names
    so it validates the Paimon payload in both 4.1.3 and 4.1.5 packages.
    
    ### Details
    
    - **Affected:** docker/runtime/all-in-one/4.1/build.sh,
    resource/smoke-test.sh
      and the README.
    - **Developer and release impact:** The default builds both
    architectures for
    both flavors. Single-platform builds still use --platform. Local output,
    release tarballs and --test require a matching single platform.
    Multi-platform
    local builds continue to require the containerd image store; --push
    retains
      registry output support.
    - **License and third-party:** None.
    - **Rollback:** Restore the previous platform default and associated
    examples.
    
    ### Testing
    
    - bash -n and ShellCheck passed for the build script; git diff --check
    passed.
    - Eleven offline checks with a temporary Docker substitute passed:
    default
    platforms for both flavor tags, explicit amd64 and arm64 overrides, push
    without containerd, rejection of the classic store and multi-platform
    smoke
    tests, rejection of multi-platform local output and tarballs,
    single-platform
      directory builds, and the help text.
    - Built 4.1.5 base and full images for linux/amd64 and linux/arm64 using
    the
    default platform selection. Verified both platforms, version/flavor
    labels
      and the packaged smoke-test checksum.
    - resource/smoke-test.sh passed on native arm64 for 4.1.3 base, 4.1.5
    base and
      4.1.5 full: JNI payload, flavor pruning, insert/select, Stream Load,
      aggregation and schema changes. The 4.1.5 FE and BE report
      doris-4.1.5-release-917b20be838.
    - The original Paimon check failed on 4.1.5; the updated check accepts
    the
    release's paimon-connector directory and the legacy paimon-scanner
    directory.
    - bash -n and ShellCheck at warning level passed for the updated smoke
    test.
    - amd64 runtime execution was not tested on this arm64 host.
---
 docker/runtime/all-in-one/4.1/README.md            | 26 +++++++++++--------
 docker/runtime/all-in-one/4.1/build.sh             | 30 ++++++++++++++--------
 .../runtime/all-in-one/4.1/resource/smoke-test.sh  |  7 ++++-
 3 files changed, 41 insertions(+), 22 deletions(-)

diff --git a/docker/runtime/all-in-one/4.1/README.md 
b/docker/runtime/all-in-one/4.1/README.md
index b18e0fb9192..9b6e1f5f7b1 100644
--- a/docker/runtime/all-in-one/4.1/README.md
+++ b/docker/runtime/all-in-one/4.1/README.md
@@ -50,25 +50,31 @@ is enabled, because it adds the compressed blobs to the 
unpacked snapshot.
 
 ## Build
 
-A plain run builds for the host architecture only. Multi-arch needs an explicit
-`--platform`; the header the script prints says which it is doing.
+A plain run builds both `linux/amd64` and `linux/arm64`. Pass `--platform` with
+one platform for a single-architecture build; the script prints the selected
+platforms before building. Keeping both platforms locally requires the
+containerd image store; otherwise pass `--push`.
 
 ```shell
-# both tags, from the official 4.1.3 component images, host architecture only
+# both tags, from the official 4.1.3 component images, amd64 + arm64
 ./build.sh -v 4.1.3
 
-# base tag only, then smoke test it
-./build.sh -v 4.1.3 -f base -t
+# base tag for arm64, then smoke test it (use linux/amd64 on x86)
+./build.sh -v 4.1.3 -f base --platform linux/arm64 -t
 
-# from a locally built ./output instead (build.sh --fe --be --cloud; without
+# from an arm64 ./output instead (build.sh --fe --be --cloud; without
 # --cloud there is no ms/ and the image cannot serve the cloud compose file)
-./build.sh -v dev -s local
+./build.sh -v dev -s local --platform linux/arm64
 
 # from an extracted release tarball, e.g. while the component images for a
 # new release are not on Docker Hub yet
-./build.sh -v 4.1.4 -s tarball --tarball-dir ~/apache-doris-4.1.4-bin-arm64
+./build.sh -v 4.1.4 -s tarball --platform linux/arm64 --tarball-dir 
~/apache-doris-4.1.4-bin-arm64
 ```
 
+`--test` requires a single platform that runs on the test host. Local output
+and release tarballs contain binaries for one architecture, so `--source local`
+and `--source tarball` require a single `--platform` matching those binaries.
+
 `./build.sh --help` lists the rest. The build context is the repository root,
 narrowed to a few KB by `Dockerfile.dockerignore`; a local `./output` or a
 tarball directory is passed as BuildKit named contexts instead, so it can live
@@ -94,7 +100,7 @@ its own, so no per-arch directories have to be staged by 
hand.
 is foreign to the machine:
 
 ```shell
-./build.sh -v 4.1.3 --platform linux/amd64,linux/arm64 --push
+./build.sh -v 4.1.3 --push
 ```
 
 Keeping the result locally instead of pushing works only with the containerd
@@ -185,7 +191,7 @@ named as a string literal under `be/src/format_v2/jni/` and
 
 | class | directory | tag |
 |---|---|---|
-| `PaimonJniScanner` | `paimon-scanner` | both |
+| `PaimonJniScanner` | `paimon-scanner` / `paimon-connector` (depends on the 
4.1.x release) | both |
 | `IcebergSysTableJniScanner` | `iceberg-metadata-scanner` | both |
 | `JdbcJniScanner` | `jdbc-scanner` | both |
 | (`JniWriter`) | `java-writer` | both |
diff --git a/docker/runtime/all-in-one/4.1/build.sh 
b/docker/runtime/all-in-one/4.1/build.sh
index c06eefb6f87..21d7537ce10 100755
--- a/docker/runtime/all-in-one/4.1/build.sh
+++ b/docker/runtime/all-in-one/4.1/build.sh
@@ -29,7 +29,7 @@ VERSION=""
 SOURCE="image"
 FLAVORS=""
 STRIP_MODE="debug"
-PLATFORM=""
+PLATFORM="linux/amd64,linux/arm64"
 LOCAL_OUTPUT="output"
 TARBALL_DIR=""
 RUN_TEST=false
@@ -64,8 +64,10 @@ Options:
                           debug -> strip --strip-debug, keeps .symtab
                           full  -> strip -s
                           none  -> ship doris_be and doris_cloud as-is (+2.2 
GB)
-      --platform <p>    Target platform(s), e.g. linux/amd64 or
-                        linux/amd64,linux/arm64. Comma-separated values produce
+      --platform <p>    Target platform(s) (default: linux/amd64,linux/arm64).
+                        Pass one platform for a single-architecture build.
+                        Local output and tarballs need one matching platform.
+                        Comma-separated values produce
                         one multi-arch tag (an OCI image index) that resolves 
to
                         the right image per host, the way apache/doris:fe-* 
does.
                         Keeping a multi-platform result locally needs the
@@ -74,16 +76,19 @@ Options:
   -i, --image <name>    Image name without tag    (default: apache/doris)
       --no-cache        Pass --no-cache to docker build.
   -t, --test            Run resource/smoke-test.sh against each image built.
+                        Requires a single --platform that runs on this host.
   -h, --help            This message.
 
 Examples:
-  ./build.sh -v 4.1.3                     # both flavors from the official 
images
-  ./build.sh -v 4.1.3 -f base -t          # base only, then smoke test it
-  ./build.sh -v dev -s local -f full      # from a local ./output (build.sh 
--fe --be --cloud)
-  ./build.sh -v 4.1.4 -s tarball --tarball-dir ~/apache-doris-4.1.4-bin-arm64
+  ./build.sh -v 4.1.3                     # both flavors, amd64 + arm64
+  ./build.sh -v 4.1.3 -f base --platform linux/arm64 -t
+                                          # base for arm64, then smoke test it
+  ./build.sh -v dev -s local -f full --platform linux/arm64
+                                          # from an arm64 ./output (build.sh 
--fe --be --cloud)
+  ./build.sh -v 4.1.4 -s tarball --platform linux/arm64 --tarball-dir 
~/apache-doris-4.1.4-bin-arm64
                                           # from a release tarball, e.g. 
before the
                                           # component images are on Docker Hub
-  ./build.sh -v 4.1.3 --platform linux/amd64,linux/arm64 --push
+  ./build.sh -v 4.1.3 --push
                                           # one multi-arch tag for both
 
 Building a foreign architecture goes through emulation, and the two apt layers
@@ -169,7 +174,11 @@ command -v docker >/dev/null || { echo "error: docker not 
found" >&2; exit 1; }
 # nowhere to put a second architecture, so the build has to go straight to a
 # registry.
 MULTI_PLATFORM=false
-case "${PLATFORM}" in *,*) MULTI_PLATFORM=true ;; esac
+case "${PLATFORM}" in *,*) MULTI_PLATFORM=true ;; *) ;; esac
+if [[ "${MULTI_PLATFORM}" == true && -n "${ARTIFACT_DIR}" ]]; then
+    echo "error: --source ${SOURCE} contains single-architecture binaries; 
pass --platform with the matching single platform" >&2
+    exit 1
+fi
 if [[ "${MULTI_PLATFORM}" == true && "${PUSH}" != true ]]; then
     if ! docker info --format '{{range .DriverStatus}}{{.}}{{end}}' 
2>/dev/null | grep -q containerd; then
         echo "error: a multi-platform build needs --push, or the containerd 
image store" >&2
@@ -190,8 +199,7 @@ if ! docker buildx version >/dev/null 2>&1; then
     fi
 fi
 
-# Say which platforms are being built. Leaving --platform unset means the host
-# architecture only, which is easy to mistake for a multi-arch build.
+# Report the selected platforms, including the multi-architecture default.
 if [[ -n "${PLATFORM}" ]]; then
     platform_note="${PLATFORM}"
 else
diff --git a/docker/runtime/all-in-one/4.1/resource/smoke-test.sh 
b/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
index 1109db5d0fb..a87ac867ce4 100755
--- a/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
+++ b/docker/runtime/all-in-one/4.1/resource/smoke-test.sh
@@ -90,8 +90,13 @@ for d in be/lib/java_extensions/preload-extensions \
 done
 
 step "flavor payload matches ${FLAVOR}"
+# Paimon's package directory differs across 4.1.x releases.
+in_image "be/lib/java_extensions/paimon-scanner" \
+    || in_image "be/lib/java_extensions/paimon-connector" \
+    || fail "missing from base payload: paimon-scanner or paimon-connector"
+echo "  ok Paimon scanner"
 # Present in both tags.
-for d in paimon-scanner iceberg-metadata-scanner jdbc-scanner java-writer; do
+for d in iceberg-metadata-scanner jdbc-scanner java-writer; do
     in_image "be/lib/java_extensions/${d}" || fail "missing from base payload: 
${d}"
     echo "  ok ${d}"
 done


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to