This is an automated email from the ASF dual-hosted git repository.

hello-stephen pushed a commit to branch branch-4.1
in repository https://gitbox.apache.org/repos/asf/doris.git


The following commit(s) were added to refs/heads/branch-4.1 by this push:
     new 0d85479c66f branch-4.1: [chore](docker)Support rootless Docker for 
third-party environments #70457 (#70579)
0d85479c66f is described below

commit 0d85479c66f9270023d4721c73e9bb13e36d3bf8
Author: Mingyu Chen (Rayner) <[email protected]>
AuthorDate: Sun Nov 15 14:16:30 2026 +0800

    branch-4.1: [chore](docker)Support rootless Docker for third-party 
environments #70457 (#70579)
    
    bp #70457
    
    ### Conflict resolution for branch-4.1
    
    #70457 does not apply cleanly here: it builds on launcher wrappers that
    master got from #64304, and it touches the Fluss fixture, which
    branch-4.1 does not have. What differs from the original:
    
    - **Fluss:** branch-4.1 has no Fluss docker environment, so
    `docker-compose/fluss/README.md` and `fluss.env.tpl` stay deleted,
    `fluss-network.sh` and `test/fluss-network-test.sh` are left out, and
    the Fluss sentence in `docker/thirdparties/README.md` and the Fluss case
    in `test/launcher-stop-test.sh` are dropped.
    - **`run-thirdparties-docker.sh`:** branch-4.1 had no `docker_cli` /
    `host_admin_cmd` wrappers, so besides the conflicted hunks, the 21
    remaining `sudo docker` calls become `docker_cli` (as on master), the
    Lance jar download and the Iceberg data extraction drop `sudo`, and
    LakeSoul's `sudo rm -rf data` becomes `reset_data_dirs` plus `rmdir`,
    the pattern #70457 uses for Iceberg. `iceberg-rest` is added to
    `ALL_COMPONENTS`. Master-only code (MongoDB, Fluss, the Iceberg jar
    check) is not added. The launcher has no `sudo` left.
    - **`ranger_settings.env`:** takes #70457's overridable form. This also
    renames branch-4.1's `RANGER_SOLAR_PORT` to `RANGER_SOLR_PORT`, the name
    `ranger.yaml.tpl` already reads (master fixed it in #66770).
    - **`test/launcher-stop-test.sh`:** branch-4.1's `custom_settings.env`
    sets `CONTAINER_UID` unconditionally, so the test rewrites the copied
    file the way CI does.
    
    The other 25 files carry #70457's changes unchanged.
    
    ### Testing
    
    - `bash -n` passes on every `docker/thirdparties/*.sh`.
    `test/docker-command-test.sh`, `test/snapshot-hive-baseline-test.sh` and
    `test/launcher-stop-test.sh` pass locally (macOS, bash 5.3, GNU getopt
    as on Linux CI); the stop test fails when the `STOP` guard on Hive data
    preparation is removed.
    - Not run: the launcher against a real rootless Docker daemon. The
    External run on this PR is the end-to-end check; External's preflight on
    branch-4.1 requires this pick, and #70578 makes `TeamCity/external`
    required here.
    
    ### AI Disclosure
    
    - Tool: Claude Code (Claude Opus 5.5)
    - Generated: the conflict resolution and the adaptations above, and this
    description
    - Human review: not stated
    
    ---------
    
    Co-authored-by: zhangstar333 <[email protected]>
    Co-authored-by: Dongyang Li <[email protected]>Co-Authored-By: Claude 
Opus 5.5 <[email protected]>
    Co-authored-by: lidongyang <[email protected]>
---
 docker/thirdparties/README.md                      |  71 +++++++
 docker/thirdparties/docker-command.sh              |  92 +++++++++
 .../thirdparties/docker-compose/db2/db2.yaml.tpl   |   5 +-
 .../scripts/init-db2.sh}                           |  23 ++-
 .../docker-compose/elasticsearch/es.yaml.tpl       |   2 +
 .../elasticsearch/scripts/es_init.sh               |   3 +
 docker/thirdparties/docker-compose/hive/README.md  |   4 +-
 .../thirdparties/docker-compose/hive/README_ZH.md  |   4 +-
 .../hive/scripts/snapshot-hive-baseline.sh         |  31 ++-
 .../thirdparties/docker-compose/hudi/hudi.env.tpl  |   2 +-
 .../thirdparties/docker-compose/hudi/hudi.yaml.tpl |  24 ++-
 .../iceberg-rest/docker-compose.yaml.tpl           |  10 +-
 .../iceberg-rest/hdfs-rootless.yaml.tpl            |  44 +++++
 .../docker-compose/kafka/kafka.yaml.tpl            |   5 +-
 .../docker-compose/kerberos/kerberos.yaml.tpl      |   6 +
 .../minio/minio-RELEASE.2024-11-07.yaml.tpl        |   1 -
 .../docker-compose/mysql/mysql-5.7.yaml.tpl        |   5 +-
 .../docker-compose/oracle/oracle-11.yaml.tpl       |   9 +-
 .../docker-compose/polaris/docker-compose.yaml.tpl |   5 +
 .../docker-compose/polaris/init-catalog.sh         |   2 +
 .../docker-compose/polaris/polaris_settings.env    |   8 +-
 .../postgresql/postgresql-14.yaml.tpl              |   5 +-
 .../docker-compose/ranger/ranger_settings.env      |   6 +-
 .../docker-compose/sqlserver/sqlserver.yaml.tpl    |  20 +-
 docker/thirdparties/docker-health.sh               |   4 +-
 docker/thirdparties/run-thirdparties-docker.sh     | 217 ++++++++++-----------
 docker/thirdparties/test/docker-command-test.sh    | 138 +++++++++++++
 docker/thirdparties/test/launcher-stop-test.sh     |  79 ++++++++
 .../test/snapshot-hive-baseline-test.sh            |  99 ++++++++++
 29 files changed, 756 insertions(+), 168 deletions(-)

diff --git a/docker/thirdparties/README.md b/docker/thirdparties/README.md
new file mode 100644
index 00000000000..507e296d937
--- /dev/null
+++ b/docker/thirdparties/README.md
@@ -0,0 +1,71 @@
+<!--
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+
+  http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License.
+-->
+
+# Thirdparty Docker environments
+
+Run a component with Bash 4 or newer and Docker Compose v2:
+
+```bash
+bash run-thirdparties-docker.sh -c iceberg
+bash run-thirdparties-docker.sh -c iceberg --stop
+```
+
+## Rootless Docker on Linux
+
+Install and start the current user's rootless Docker daemon first. Its setup
+requires subordinate UID/GID ranges, `newuidmap`/`newgidmap`, and enabled
+unprivileged user namespaces. Once configured, launching these environments
+does not require host `sudo` or membership in the host `docker` group.
+
+On Linux, the launcher requires a rootless daemon and rejects rootful daemons.
+It detects the user's `$XDG_RUNTIME_DIR/docker.sock` socket unless an endpoint
+or context was explicitly selected. Docker commands always run as the current
+user; there is no automatic elevation or `DOCKER_USE_SUDO` switch.
+The endpoint can also be selected explicitly:
+
+```bash
+export DOCKER_HOST="unix://${XDG_RUNTIME_DIR:-/run/user/$(id -u)}/docker.sock"
+docker info --format '{{json .SecurityOptions}}' # must include name=rootless
+bash run-thirdparties-docker.sh -c iceberg
+```
+
+On macOS, the launcher uses the current user's Docker Desktop endpoint.
+
+Keep the working directory writable by the current user. The launcher cleans
+fixture files owned by subordinate container UIDs from inside the same rootless
+user namespace, using `alpine:3.20`. It does not change host ownership to root.
+
+Hive, Kerberos and Iceberg REST's built-in HDFS require Docker Engine 29.5 or
+newer for rootless host networking. Hive defaults to 
`HIVE_HOST_ALIAS=localhost`
+and uses loopback for its same-host services. Iceberg REST HDFS uses
+host ports 20020, 20870, 20864, 20866 and
+20867, so its DataNode block addresses are reachable by host clients.
+
+Kerberos client configuration and keytabs remain under
+`docker-compose/kerberos/two-kerberos-hives/`; use the printed `KRB5_CONFIG`
+path for host clients. The rootless launcher does not write `/etc/hosts`,
+`/etc/krb5.conf` or `/keytabs`. The DB2 image retains its container-level
+`privileged` setting within the rootless daemon's user namespace.
+
+`--reserve-ports` changes a host sysctl and is unavailable in rootless mode.
+It is optional; an operator can reserve ports separately when required.
+Image pulls and fixture downloads still require access to their registries
+and artifact repositories. `--no-load-data` skips Hive fixture refresh;
+combine it with `--hive-mode rebuild` to validate a fresh Hive installation
+without downloading a regression-data baseline.
diff --git a/docker/thirdparties/docker-command.sh 
b/docker/thirdparties/docker-command.sh
new file mode 100755
index 00000000000..ffd4533cd3e
--- /dev/null
+++ b/docker/thirdparties/docker-command.sh
@@ -0,0 +1,92 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+configure_docker_access() {
+    local rootless_runtime_dir
+    local rootless_socket
+    local security_options
+
+    rootless_runtime_dir="${XDG_RUNTIME_DIR:-/run/user/$(id -u)}"
+    rootless_socket="${rootless_runtime_dir}/docker.sock"
+
+    # An explicit endpoint or context takes precedence over socket discovery.
+    if [[ "${HOST_OS}" != "Darwin" && -z "${DOCKER_HOST+x}" \
+        && -z "${DOCKER_CONTEXT:-}" && -S "${rootless_socket}" ]]; then
+        export DOCKER_HOST="unix://${rootless_socket}"
+    fi
+
+    if ! security_options="$(docker info --format '{{json 
.SecurityOptions}}')"; then
+        echo "Unable to access Docker as the current user; start the user's 
rootless daemon or select its endpoint" >&2
+        return 1
+    fi
+    # Docker Desktop on macOS runs its daemon in a VM. Linux requires the
+    # user's rootless daemon; an accessible system daemon is not accepted.
+    if [[ "${HOST_OS}" != "Darwin" && "${security_options}" != 
*'"name=rootless"'* ]]; then
+        echo "This launcher requires rootless Docker on Linux; the selected 
daemon is rootful" >&2
+        return 1
+    fi
+}
+
+require_rootless_host_network() {
+    local server_version
+    local major
+    local minor
+
+    if [[ "${HOST_OS}" == "Darwin" ]]; then
+        return 0
+    fi
+    server_version="$(docker_cli info --format '{{.ServerVersion}}')" || return
+    major="${server_version%%.*}"
+    minor="${server_version#*.}"
+    minor="${minor%%.*}"
+    # Older rootless host networks cannot reach ports published on the real
+    # host, which these fixtures use for PostgreSQL and HDFS RPC.
+    if ((major < 29 || (major == 29 && minor < 5))); then
+        echo "Rootless host networking requires Docker Engine 29.5 or newer 
(found ${server_version})" >&2
+        return 1
+    fi
+}
+
+rootless_fixture_cmd() {
+    local fixture_dir="$1"
+    shift
+
+    # Stay in the daemon's user namespace when images have chowned bind mounts
+    # to subordinate UIDs. Mount only the fixture directory being operated on.
+    docker_cli run --rm --network none --user 0:0 \
+        --security-opt no-new-privileges \
+        --mount "type=bind,source=${fixture_dir},target=/fixture" \
+        "${THIRDPARTY_FILESYSTEM_IMAGE:-alpine:3.20}" "$@"
+}
+
+reset_rootless_data_dir() {
+    local data_dir="$1"
+
+    mkdir -p "${data_dir}"
+    rootless_fixture_cmd "${data_dir}" sh -ec '
+            find /fixture -mindepth 1 -maxdepth 1 -exec rm -rf -- {} +
+            chown 0:0 /fixture
+            chmod 755 /fixture
+        '
+}
+
+chmod_fixture_dir() {
+    local mode="$1"
+    local fixture_dir="$2"
+    rootless_fixture_cmd "${fixture_dir}" chmod -R "${mode}" /fixture
+}
diff --git a/docker/thirdparties/docker-compose/db2/db2.yaml.tpl 
b/docker/thirdparties/docker-compose/db2/db2.yaml.tpl
index 49a09656549..c8e944d52cf 100644
--- a/docker/thirdparties/docker-compose/db2/db2.yaml.tpl
+++ b/docker/thirdparties/docker-compose/db2/db2.yaml.tpl
@@ -24,13 +24,14 @@ services:
       - ${DOCKER_DB2_EXTERNAL_PORT}:50000
     privileged: true
     healthcheck:
-      test: ["CMD-SHELL", "su - db2inst1 -c \"source ~/.bash_profile; db2 
connect to doris && db2 'select 1 from sysibm.sysdummy1'\""]
+      test: ["CMD-SHELL", "test -f /tmp/doris_db2_initialized && su - db2inst1 
-c \"source ~/.bash_profile; db2 connect to doris && db2 'select count(*) from 
doris_test.sample_table'\""]
       interval: 20s
       timeout: 60s
       retries: 30
       start_period: 300s
     volumes:
-      - ./init:/docker-entrypoint-initdb.d
+      - ./init:/docker-entrypoint-initdb.d:ro
+      - ./scripts/init-db2.sh:/var/custom/init-db2.sh:ro
     environment:
       - LICENSE=accept
       - DBNAME=doris
diff --git a/docker/thirdparties/docker-compose/ranger/ranger_settings.env 
b/docker/thirdparties/docker-compose/db2/scripts/init-db2.sh
old mode 100644
new mode 100755
similarity index 52%
copy from docker/thirdparties/docker-compose/ranger/ranger_settings.env
copy to docker/thirdparties/docker-compose/db2/scripts/init-db2.sh
index 13dd93d517d..e98334d1e97
--- a/docker/thirdparties/docker-compose/ranger/ranger_settings.env
+++ b/docker/thirdparties/docker-compose/db2/scripts/init-db2.sh
@@ -16,6 +16,23 @@
 # specific language governing permissions and limitations
 # under the License.
 
-export RANGER_SOLAR_PORT=8983
-export RANGER_PORT=6081
-export RANGER_MYSQL_PORT=33061
+set -euo pipefail
+rm -f /tmp/doris_db2_initialized
+
+# The IBM image executes /var/custom scripts after creating DBNAME. It does
+# not execute SQL files mounted at /docker-entrypoint-initdb.d by itself.
+su - "${DB2INSTANCE}" -c 'bash -se' <<'EOF'
+. "${HOME}/sqllib/db2profile"
+db2 connect to doris
+schema_count=$(db2 -x "SELECT COUNT(*) FROM SYSCAT.SCHEMATA WHERE 
SCHEMANAME='DORIS_TEST'")
+if ((schema_count == 0)); then
+    db2 -stvf /docker-entrypoint-initdb.d/02-create-schema.sql
+    db2 -stvf /docker-entrypoint-initdb.d/03-create-table.sql
+    db2 -stvf /docker-entrypoint-initdb.d/04-insert.sql
+fi
+rows=$(db2 -x 'SELECT COUNT(*) FROM doris_test.sample_table')
+[[ "${rows}" -eq 3 ]]
+db2 connect reset
+EOF
+
+touch /tmp/doris_db2_initialized
diff --git a/docker/thirdparties/docker-compose/elasticsearch/es.yaml.tpl 
b/docker/thirdparties/docker-compose/elasticsearch/es.yaml.tpl
index 83303c250a0..f838af7e829 100644
--- a/docker/thirdparties/docker-compose/elasticsearch/es.yaml.tpl
+++ b/docker/thirdparties/docker-compose/elasticsearch/es.yaml.tpl
@@ -127,6 +127,8 @@ services:
       ES_8_HOST: "doris--es_8"
     command: [ "sh", "-c", "/mnt/scripts/es_init.sh" ]
     depends_on:
+      doris--es_5:
+        condition: service_healthy
       doris--es_6:
         condition: service_healthy
       doris--es_7:
diff --git 
a/docker/thirdparties/docker-compose/elasticsearch/scripts/es_init.sh 
b/docker/thirdparties/docker-compose/elasticsearch/scripts/es_init.sh
index 51f81ced938..d8e31323b00 100755
--- a/docker/thirdparties/docker-compose/elasticsearch/scripts/es_init.sh
+++ b/docker/thirdparties/docker-compose/elasticsearch/scripts/es_init.sh
@@ -16,6 +16,8 @@
 # specific language governing permissions and limitations
 # under the License.
 
+set -eo pipefail
+
 generate_bulk_request() {
     local index_name=$1
     local type_value=$2
@@ -234,3 +236,4 @@ generate_bulk_request "test_keyword_array" "" "item_" 
"$keyword_array_data_file"
 curl -X POST "http://${ES_8_HOST}:9200/_bulk"; --data-binary 
"@$keyword_array_bulk_file" -H "Content-Type: application/json"
 
 touch /tmp/SUCCESS
+exec tail -f /dev/null
diff --git a/docker/thirdparties/docker-compose/hive/README.md 
b/docker/thirdparties/docker-compose/hive/README.md
index 57c6d414b1d..65b8266217b 100644
--- a/docker/thirdparties/docker-compose/hive/README.md
+++ b/docker/thirdparties/docker-compose/hive/README.md
@@ -116,11 +116,11 @@ Relevant env vars:
 After bootstrapping a clean Hive stack, stop the containers and run:
 
 ```bash
-sudo docker compose -p "${CONTAINER_UID}hive3" \
+docker compose -p "${CONTAINER_UID}hive3" \
   -f docker/thirdparties/docker-compose/hive/hive-3x.yaml down
 
 bash docker/thirdparties/docker-compose/hive/scripts/snapshot-hive-baseline.sh 
\
-  "${CONTAINER_UID}hive3" /tmp/hive3-baseline.tar.gz
+  "doris-shared-hive3" /tmp/hive3-baseline.tar.gz
 ```
 
 Then upload the resulting tarball to OSS at 
`oss://<s3BucketName>/regression/datalake/pipeline_data/hive_baseline/hive3-baseline-<version>.tar.gz`
 (same convention for `hive2`).
diff --git a/docker/thirdparties/docker-compose/hive/README_ZH.md 
b/docker/thirdparties/docker-compose/hive/README_ZH.md
index 8cc32162dc6..546eaf3b8e5 100644
--- a/docker/thirdparties/docker-compose/hive/README_ZH.md
+++ b/docker/thirdparties/docker-compose/hive/README_ZH.md
@@ -116,11 +116,11 @@ Hive 运行态(HDFS 数据、Postgres Metastore、模块 SHA 记录)存放
 在一次完整 bootstrap 成功后,停止容器并运行:
 
 ```bash
-sudo docker compose -p "${CONTAINER_UID}hive3" \
+docker compose -p "${CONTAINER_UID}hive3" \
   -f docker/thirdparties/docker-compose/hive/hive-3x.yaml down
 
 bash docker/thirdparties/docker-compose/hive/scripts/snapshot-hive-baseline.sh 
\
-  "${CONTAINER_UID}hive3" /tmp/hive3-baseline.tar.gz
+  "doris-shared-hive3" /tmp/hive3-baseline.tar.gz
 ```
 
 然后把得到的 tarball 上传到 
`oss://<s3BucketName>/regression/datalake/pipeline_data/hive_baseline/hive3-baseline-<version>.tar.gz`(`hive2`
 同理)。
diff --git 
a/docker/thirdparties/docker-compose/hive/scripts/snapshot-hive-baseline.sh 
b/docker/thirdparties/docker-compose/hive/scripts/snapshot-hive-baseline.sh
index edbe2788dcc..e8024398a2b 100755
--- a/docker/thirdparties/docker-compose/hive/scripts/snapshot-hive-baseline.sh
+++ b/docker/thirdparties/docker-compose/hive/scripts/snapshot-hive-baseline.sh
@@ -24,7 +24,7 @@ set -eo pipefail
 #   snapshot-hive-baseline.sh <volume_prefix> <output_path>
 #
 # Example:
-#   bash snapshot-hive-baseline.sh doris--syt--hive3 /tmp/hive3-baseline.tar.gz
+#   bash snapshot-hive-baseline.sh doris-shared-hive3 
/tmp/hive3-baseline.tar.gz
 #
 # Prerequisites:
 #   - All Hive containers must be STOPPED before running this script.
@@ -33,35 +33,48 @@ set -eo pipefail
 VOLUME_PREFIX="${1:?Usage: $0 <volume_prefix> <output_path>}"
 OUTPUT_PATH="${2:?Usage: $0 <volume_prefix> <output_path>}"
 
+THIRDPARTY_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../../.." &>/dev/null && 
pwd)"
+HOST_OS="$(uname -s)"
+# shellcheck source=../../../docker-command.sh
+. "${THIRDPARTY_ROOT}/docker-command.sh"
+configure_docker_access
+
+docker_cli() {
+    docker "$@"
+}
+
 echo "[snapshot] volume prefix: ${VOLUME_PREFIX}"
 echo "[snapshot] output: ${OUTPUT_PATH}"
 
 # Verify all 4 volumes exist
 for suffix in namenode datanode pgdata state; do
     vol="${VOLUME_PREFIX}-${suffix}"
-    if ! sudo docker volume inspect "${vol}" >/dev/null 2>&1; then
+    if ! docker_cli volume inspect "${vol}" >/dev/null 2>&1; then
         echo "ERROR: volume ${vol} does not exist" >&2
         exit 1
     fi
 done
 
 _t0=$(date +%s)
+tmp_output="$(mktemp "${OUTPUT_PATH}.XXXXXX")"
+trap 'rm -f "${tmp_output}"' EXIT
 
 # Mount all 4 volumes read-only into a single alpine container and tar them
 # in one pass. Copy only the small state volume to ephemeral container storage
 # so legacy baseline.version files can be dropped from newly exported tarballs.
-sudo docker run --rm \
-    -v "${VOLUME_PREFIX}-namenode:/snapshot/namenode:ro" \
-    -v "${VOLUME_PREFIX}-datanode:/snapshot/datanode:ro" \
-    -v "${VOLUME_PREFIX}-pgdata:/snapshot/pgdata:ro" \
+docker_cli run --rm --network none --security-opt no-new-privileges \
+    -v "${VOLUME_PREFIX}-namenode:/work/namenode:ro" \
+    -v "${VOLUME_PREFIX}-datanode:/work/datanode:ro" \
+    -v "${VOLUME_PREFIX}-pgdata:/work/pgdata:ro" \
     -v "${VOLUME_PREFIX}-state:/snapshot/state:ro" \
-    alpine sh -c '
+    "${THIRDPARTY_FILESYSTEM_IMAGE:-alpine:3.20}" sh -ec '
         mkdir -p /work/state
         cp -a /snapshot/state/. /work/state/
         rm -f /work/state/baseline.version
-        tar czf - -C /snapshot namenode datanode pgdata -C /work state
+        tar czf - -C /work namenode datanode pgdata state
     ' \
-    > "${OUTPUT_PATH}"
+    > "${tmp_output}"
+mv "${tmp_output}" "${OUTPUT_PATH}"
 
 size=$(du -h "${OUTPUT_PATH}" | cut -f1)
 echo "[snapshot] done took=$(( $(date +%s) - _t0 ))s size=${size}"
diff --git a/docker/thirdparties/docker-compose/hudi/hudi.env.tpl 
b/docker/thirdparties/docker-compose/hudi/hudi.env.tpl
index e04012010e9..69eefee256c 100644
--- a/docker/thirdparties/docker-compose/hudi/hudi.env.tpl
+++ b/docker/thirdparties/docker-compose/hudi/hudi.env.tpl
@@ -15,7 +15,7 @@
 # limitations under the License.
 #
 
-CONTAINER_UID=doris--
+CONTAINER_UID=${CONTAINER_UID}
 HUDI_NETWORK=${CONTAINER_UID}hudi-network
 
 # Ports exposed to host
diff --git a/docker/thirdparties/docker-compose/hudi/hudi.yaml.tpl 
b/docker/thirdparties/docker-compose/hudi/hudi.yaml.tpl
index 94cd3adb3e0..554e77f033d 100644
--- a/docker/thirdparties/docker-compose/hudi/hudi.yaml.tpl
+++ b/docker/thirdparties/docker-compose/hudi/hudi.yaml.tpl
@@ -27,13 +27,18 @@ services:
   ${CONTAINER_UID}hudi-minio:
     image: doristhirdpartydocker/minio:RELEASE.2025-01-20T14-49-07Z
     container_name: ${CONTAINER_UID}hudi-minio
-    command: server /data --console-address ":${MINIO_CONSOLE_PORT}"
+    command: server /data --console-address ":9001"
     environment:
       MINIO_ROOT_USER: ${MINIO_ROOT_USER}
       MINIO_ROOT_PASSWORD: ${MINIO_ROOT_PASSWORD}
     ports:
       - "${MINIO_API_PORT}:9000"
       - "${MINIO_CONSOLE_PORT}:9001"
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/ready";]
+      interval: 5s
+      timeout: 10s
+      retries: 60
     networks:
       - ${HUDI_NETWORK}
 
@@ -43,13 +48,13 @@ services:
     entrypoint: |
       /bin/bash -c "
       set -euo pipefail
-      sleep 5
       mc alias set myminio http://${CONTAINER_UID}hudi-minio:9000 
${MINIO_ROOT_USER} ${MINIO_ROOT_PASSWORD}
-      mc mb --quiet myminio/${HUDI_BUCKET} || true
-      mc mb --quiet myminio/${HUDI_BUCKET}-tmp || true
+      mc mb --quiet --ignore-existing myminio/${HUDI_BUCKET}
+      mc mb --quiet --ignore-existing myminio/${HUDI_BUCKET}-tmp
       "
     depends_on:
-      - ${CONTAINER_UID}hudi-minio
+      ${CONTAINER_UID}hudi-minio:
+        condition: service_healthy
     networks:
       - ${HUDI_NETWORK}
 
@@ -123,9 +128,12 @@ services:
       - ./scripts:/opt/hudi-scripts
       - ./cache:/opt/hudi-cache
     depends_on:
-      - ${CONTAINER_UID}hudi-minio
-      - ${CONTAINER_UID}hudi-minio-mc
-      - ${CONTAINER_UID}hudi-metastore
+      ${CONTAINER_UID}hudi-minio:
+        condition: service_started
+      ${CONTAINER_UID}hudi-minio-mc:
+        condition: service_completed_successfully
+      ${CONTAINER_UID}hudi-metastore:
+        condition: service_started
     command: ["/opt/hudi-scripts/init.sh"]
     ports:
       - "${SPARK_UI_PORT}:8080"
diff --git 
a/docker/thirdparties/docker-compose/iceberg-rest/docker-compose.yaml.tpl 
b/docker/thirdparties/docker-compose/iceberg-rest/docker-compose.yaml.tpl
index ee70b27e3f8..e8eef3ec2b0 100644
--- a/docker/thirdparties/docker-compose/iceberg-rest/docker-compose.yaml.tpl
+++ b/docker/thirdparties/docker-compose/iceberg-rest/docker-compose.yaml.tpl
@@ -170,8 +170,8 @@ services:
       - 
CORE_CONF_io_compression_codecs=org.apache.hadoop.io.compress.SnappyCodec
       - HDFS_CONF_dfs_webhdfs_enabled=true
       - HDFS_CONF_dfs_permissions_enabled=false
-      - HDFS_CONF_dfs_namenode_rpc_address=hdfs-namenode:8020
-      - HDFS_CONF_dfs_namenode_http_address=0.0.0.0:9870
+      - HDFS_CONF_dfs_namenode_rpc___address=hdfs-namenode:8020
+      - HDFS_CONF_dfs_namenode_http___address=0.0.0.0:9870
       - HDFS_CONF_dfs_replication=1
       - HDFS_CONF_dfs_namenode_name_dir=/hadoop/dfs/name
       - 
HDFS_CONF_dfs_namenode_datanode_registration_ip___hostname___check=false
@@ -226,6 +226,12 @@ services:
     depends_on:
       - hdfs-namenode
       - hdfs-datanode
+    healthcheck:
+      # This image has bash but does not include curl or wget.
+      test: ["CMD", "bash", "-ec", "exec 3<>/dev/tcp/127.0.0.1/8181; printf 
'GET /v1/config HTTP/1.0\\r\\nHost: localhost\\r\\n\\r\\n' >&3; head -n 1 <&3 | 
grep -q '^HTTP/1.[01] 200 '"]
+      interval: 10s
+      timeout: 5s
+      retries: 5
 
 networks:
   ${CONTAINER_UID}iceberg-rest:
diff --git 
a/docker/thirdparties/docker-compose/iceberg-rest/hdfs-rootless.yaml.tpl 
b/docker/thirdparties/docker-compose/iceberg-rest/hdfs-rootless.yaml.tpl
new file mode 100644
index 00000000000..2c85f826c35
--- /dev/null
+++ b/docker/thirdparties/docker-compose/iceberg-rest/hdfs-rootless.yaml.tpl
@@ -0,0 +1,44 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Use distinct real host ports so HDFS block locations work for host clients
+# as well as the REST catalog containers. Requires rootless Docker >= 29.5.
+services:
+  hdfs-namenode:
+    network_mode: host
+    networks: !reset []
+    ports: !reset []
+    environment:
+      - CORE_CONF_fs_defaultFS=hdfs://${LOCAL_IP}:${HDFS_NAMENODE_RPC_PORT}
+      - 
HDFS_CONF_dfs_namenode_rpc___address=${LOCAL_IP}:${HDFS_NAMENODE_RPC_PORT}
+      - 
HDFS_CONF_dfs_namenode_http___address=0.0.0.0:${HDFS_NAMENODE_WEBUI_PORT}
+    healthcheck:
+      test: ["CMD", "curl", "-f", 
"http://127.0.0.1:${HDFS_NAMENODE_WEBUI_PORT}/";]
+
+  hdfs-datanode:
+    network_mode: host
+    networks: !reset []
+    ports: !reset []
+    environment:
+      - CORE_CONF_fs_defaultFS=hdfs://${LOCAL_IP}:${HDFS_NAMENODE_RPC_PORT}
+      - HDFS_CONF_dfs_datanode_hostname=${LOCAL_IP}
+      - HDFS_CONF_dfs_datanode_address=0.0.0.0:20866
+      - HDFS_CONF_dfs_datanode_ipc_address=0.0.0.0:20867
+      - HDFS_CONF_dfs_datanode_http_address=0.0.0.0:${HDFS_DATANODE_WEBUI_PORT}
+      - SERVICE_PRECONDITION=${LOCAL_IP}:${HDFS_NAMENODE_WEBUI_PORT}
+    healthcheck:
+      test: ["CMD", "curl", "-f", 
"http://127.0.0.1:${HDFS_DATANODE_WEBUI_PORT}/";]
diff --git a/docker/thirdparties/docker-compose/kafka/kafka.yaml.tpl 
b/docker/thirdparties/docker-compose/kafka/kafka.yaml.tpl
index e3322ce2ff6..777f744b8b6 100644
--- a/docker/thirdparties/docker-compose/kafka/kafka.yaml.tpl
+++ b/docker/thirdparties/docker-compose/kafka/kafka.yaml.tpl
@@ -31,8 +31,7 @@ services:
         ports:
             - ${DOCKER_ZOOKEEPER_EXTERNAL_PORT}:2181
         healthcheck:
-            # 
https://github.com/bitnami/charts/blob/62399ed3863e70775d66f9581e28129026a86e5d/bitnami/zookeeper/templates/statefulset.yaml#L404
-            test: /bin/bash -ec ZOO_HC_TIMEOUT=10 
/opt/bitnami/scripts/zookeeper/healthcheck.sh
+            test: ["CMD", "/opt/bitnami/zookeeper/bin/zkServer.sh", "status"]
             start_period: 10s
             interval: 5s
             timeout: 60s
@@ -63,7 +62,5 @@ services:
             - KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:19193
             - KAFKA_ZOOKEEPER_CONNECT=doris--zookeeper:2181
             - ALLOW_PLAINTEXT_LISTENER=yes
-        volumes:
-            - /var/run/docker.sock:/var/run/docker.sock
         networks:
             - doris--kafka--network
diff --git a/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl 
b/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
index cf1dc94867c..2a749642d32 100644
--- a/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
+++ b/docker/thirdparties/docker-compose/kerberos/kerberos.yaml.tpl
@@ -24,6 +24,12 @@ x-kerberos-image: &kerberos-image
 x-kerberos-service: &kerberos-service
   <<: *kerberos-image
   network_mode: "host"
+  extra_hosts:
+    - "hadoop-master:${IP_HOST}"
+    - "hadoop-master-2:${IP_HOST}"
+    # Hadoop's Kerberos resolver performs absolute (trailing-dot) lookups.
+    - "hadoop-master.:${IP_HOST}"
+    - "hadoop-master-2.:${IP_HOST}"
   mem_limit: 1536m
   stop_grace_period: 30s
 
diff --git 
a/docker/thirdparties/docker-compose/minio/minio-RELEASE.2024-11-07.yaml.tpl 
b/docker/thirdparties/docker-compose/minio/minio-RELEASE.2024-11-07.yaml.tpl
index 81ddd8ba027..6dfad86b97a 100644
--- a/docker/thirdparties/docker-compose/minio/minio-RELEASE.2024-11-07.yaml.tpl
+++ b/docker/thirdparties/docker-compose/minio/minio-RELEASE.2024-11-07.yaml.tpl
@@ -23,7 +23,6 @@ services:
     restart: always
     ports:
       - ${DOCKER_MINIO_EXTERNAL_PORT}:9000
-    privileged: true
     healthcheck:
       test: [ "CMD", "bash", "-c", "mc ping --count 1 local" ]
       interval: 20s
diff --git a/docker/thirdparties/docker-compose/mysql/mysql-5.7.yaml.tpl 
b/docker/thirdparties/docker-compose/mysql/mysql-5.7.yaml.tpl
index 9fe7b1a38ef..d77c52394ed 100644
--- a/docker/thirdparties/docker-compose/mysql/mysql-5.7.yaml.tpl
+++ b/docker/thirdparties/docker-compose/mysql/mysql-5.7.yaml.tpl
@@ -32,6 +32,9 @@ services:
       - bash
       - -c
       - |
+        set -e
+        mkdir -p /etc/mysql/certs
+        cp /etc/mysql/certs-source/* /etc/mysql/certs/
         chown mysql:mysql /etc/mysql/certs/*
         chmod 600 /etc/mysql/certs/server.key
         chmod 644 /etc/mysql/certs/server.crt /etc/mysql/certs/root.crt
@@ -52,7 +55,7 @@ services:
       - ./data/:/var/lib/mysql
       - ./init:/docker-entrypoint-initdb.d
       - ./my.cnf:/etc/mysql/conf.d/my.cnf
-      - ./certs:/etc/mysql/certs
+      - ./certs:/etc/mysql/certs-source:ro
     networks:
       - doris--mysql_57
 
diff --git a/docker/thirdparties/docker-compose/oracle/oracle-11.yaml.tpl 
b/docker/thirdparties/docker-compose/oracle/oracle-11.yaml.tpl
index 09314c9dc23..087833bb649 100644
--- a/docker/thirdparties/docker-compose/oracle/oracle-11.yaml.tpl
+++ b/docker/thirdparties/docker-compose/oracle/oracle-11.yaml.tpl
@@ -23,9 +23,14 @@ services:
     restart: always
     ports:
       - ${DOCKER_ORACLE_EXTERNAL_PORT}:1521
-    privileged: true
     healthcheck:
-      test: [ "CMD", "bash", "-c", "echo 'SELECT 1 FROM doris_test.deadline;' 
| ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe 
/u01/app/oracle/product/11.2.0/xe/bin/sqlplus -s DORIS_TEST/123456@localhost" ]
+      test:
+        - CMD-SHELL
+        - >-
+          printf '%s\n' 'WHENEVER SQLERROR EXIT FAILURE' 'WHENEVER OSERROR 
EXIT FAILURE'
+          'SELECT COUNT(*) FROM doris_test.deadline;' 'EXIT;'
+          | ORACLE_HOME=/u01/app/oracle/product/11.2.0/xe
+          /u01/app/oracle/product/11.2.0/xe/bin/sqlplus -L -s 
DORIS_TEST/123456@localhost
       interval: 20s
       timeout: 60s
       retries: 120
diff --git a/docker/thirdparties/docker-compose/polaris/docker-compose.yaml.tpl 
b/docker/thirdparties/docker-compose/polaris/docker-compose.yaml.tpl
index 98124890872..78c941cf03b 100644
--- a/docker/thirdparties/docker-compose/polaris/docker-compose.yaml.tpl
+++ b/docker/thirdparties/docker-compose/polaris/docker-compose.yaml.tpl
@@ -113,6 +113,11 @@ services:
     volumes:
       - ./init-catalog.sh:/init-catalog.sh:ro
     entrypoint: ["/bin/sh","-c","/init-catalog.sh"]
+    healthcheck:
+      test: ["CMD-SHELL", "test -f /tmp/polaris_initialized"]
+      interval: 5s
+      timeout: 3s
+      retries: 60
     networks:
       - ${CONTAINER_UID}polaris
 
diff --git a/docker/thirdparties/docker-compose/polaris/init-catalog.sh 
b/docker/thirdparties/docker-compose/polaris/init-catalog.sh
index 124e41845be..6e0b5a9fb0b 100755
--- a/docker/thirdparties/docker-compose/polaris/init-catalog.sh
+++ b/docker/thirdparties/docker-compose/polaris/init-catalog.sh
@@ -142,4 +142,6 @@ fi
 
 echo "[polaris-init] Permissions setup completed."
 echo "[polaris-init] Done."
+touch /tmp/polaris_initialized
+exec tail -f /dev/null
 
diff --git a/docker/thirdparties/docker-compose/polaris/polaris_settings.env 
b/docker/thirdparties/docker-compose/polaris/polaris_settings.env
index 70ba1a96b93..2242db926d4 100644
--- a/docker/thirdparties/docker-compose/polaris/polaris_settings.env
+++ b/docker/thirdparties/docker-compose/polaris/polaris_settings.env
@@ -17,15 +17,15 @@
 # under the License.
 
 # Polaris uses 20181-20182 for S3
-export POLARIS_S3_PORT=20181
-export POLARIS_S3_ADMIN_PORT=20182
+export POLARIS_S3_PORT="${POLARIS_S3_PORT:-20181}"
+export POLARIS_S3_ADMIN_PORT="${POLARIS_S3_ADMIN_PORT:-20182}"
 
 # Polaris authentication
 export POLARIS_BOOTSTRAP_PASSWORD="secret123"
 
 # MinIO (S3-compatible) configuration for local development
-export MINIO_API_PORT=20001
-export MINIO_CONSOLE_PORT=20002
+export MINIO_API_PORT="${POLARIS_MINIO_API_PORT:-20001}"
+export MINIO_CONSOLE_PORT="${POLARIS_MINIO_CONSOLE_PORT:-20002}"
 export MINIO_ACCESS_KEY="admin"
 export MINIO_SECRET_KEY="password"
 export MINIO_BUCKET="warehouse"
diff --git 
a/docker/thirdparties/docker-compose/postgresql/postgresql-14.yaml.tpl 
b/docker/thirdparties/docker-compose/postgresql/postgresql-14.yaml.tpl
index 54758285290..a8d1233c1f1 100644
--- a/docker/thirdparties/docker-compose/postgresql/postgresql-14.yaml.tpl
+++ b/docker/thirdparties/docker-compose/postgresql/postgresql-14.yaml.tpl
@@ -29,6 +29,9 @@ services:
       - bash
       - -c
       - |
+        set -e
+        mkdir -p /var/lib/postgresql/certs
+        cp /var/lib/postgresql/certs-source/* /var/lib/postgresql/certs/
         chmod 600 /var/lib/postgresql/certs/server.key
         chmod 644 /var/lib/postgresql/certs/server.crt
         chmod 644 /var/lib/postgresql/certs/root.crt
@@ -59,7 +62,7 @@ services:
     volumes:
       - ./data/data/:/var/lib/postgresql/data/
       - ./init:/docker-entrypoint-initdb.d
-      - ./certs:/var/lib/postgresql/certs
+      - ./certs:/var/lib/postgresql/certs-source:ro
     networks:
       - doris--postgres
 
diff --git a/docker/thirdparties/docker-compose/ranger/ranger_settings.env 
b/docker/thirdparties/docker-compose/ranger/ranger_settings.env
index 13dd93d517d..08c11de6495 100644
--- a/docker/thirdparties/docker-compose/ranger/ranger_settings.env
+++ b/docker/thirdparties/docker-compose/ranger/ranger_settings.env
@@ -16,6 +16,6 @@
 # specific language governing permissions and limitations
 # under the License.
 
-export RANGER_SOLAR_PORT=8983
-export RANGER_PORT=6081
-export RANGER_MYSQL_PORT=33061
+export RANGER_SOLR_PORT="${RANGER_SOLR_PORT:-8983}"
+export RANGER_PORT="${RANGER_PORT:-6081}"
+export RANGER_MYSQL_PORT="${RANGER_MYSQL_PORT:-33061}"
diff --git a/docker/thirdparties/docker-compose/sqlserver/sqlserver.yaml.tpl 
b/docker/thirdparties/docker-compose/sqlserver/sqlserver.yaml.tpl
index c2195d2f022..b14707155de 100644
--- a/docker/thirdparties/docker-compose/sqlserver/sqlserver.yaml.tpl
+++ b/docker/thirdparties/docker-compose/sqlserver/sqlserver.yaml.tpl
@@ -23,7 +23,7 @@ services:
     ports:
       - ${DOCKER_SQLSERVER_EXTERNAL_PORT}:1433
     healthcheck:
-      test: ["CMD", "/opt/mssql-tools/bin/sqlcmd", "-Usa", "-PDoris123456", 
"-Q", "select 1"]
+      test: ["CMD-SHELL", "test -f /tmp/doris_sqlserver_initialized && 
/opt/mssql-tools/bin/sqlcmd -Usa -PDoris123456 -b -Q 'SELECT 1 FROM 
doris_test.dbo.student'"]
       interval: 5s
       timeout: 30s
       retries: 120
@@ -33,26 +33,26 @@ services:
       - /bin/bash
       - -c
       - |
+        set -e
+        rm -f /tmp/doris_sqlserver_initialized
         # Launch MSSQL and send to background
         /opt/mssql/bin/sqlservr &
+        sqlserver_pid=$$!
+        trap 'kill -TERM "$$sqlserver_pid"; wait "$$sqlserver_pid"' TERM INT
         # Wait for it to be available
         echo "Waiting for MS SQL to be available ⏳"
-        /opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P 
Doris123456 -Q "SET NOCOUNT ON SELECT \"YAY WE ARE UP\" , @@servername"
-        is_up=$$?
-        while [ $$is_up -ne 0 ] ; do
-          echo -e $$(date)
-          /opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -V1 -U sa -P 
Doris123456 -Q "SET NOCOUNT ON SELECT \"YAY WE ARE UP\" , @@servername"
-          is_up=$$?
+        until /opt/mssql-tools/bin/sqlcmd -l 30 -S localhost -h-1 -b -U sa -P 
Doris123456 -Q "SET NOCOUNT ON SELECT \"YAY WE ARE UP\" , @@servername"; do
+          kill -0 "$$sqlserver_pid"
           sleep 5
         done
         # Run every script in /scripts
         # TODO set a flag so that this is only done once on creation,
         #      and not every time the container runs
         for foo in /docker-entrypoint-initdb.d/*.sql
-          do /opt/mssql-tools/bin/sqlcmd -U sa -P Doris123456 -l 30 -e -i $$foo
+          do /opt/mssql-tools/bin/sqlcmd -U sa -P Doris123456 -b -l 30 -e -i 
"$$foo"
         done
-        # So that the container doesn't shut down, sleep this thread
-        sleep infinity
+        touch /tmp/doris_sqlserver_initialized
+        wait "$$sqlserver_pid"
 
     restart: always
     environment:
diff --git a/docker/thirdparties/docker-health.sh 
b/docker/thirdparties/docker-health.sh
index 9148465a54d..7eca1e6f30a 100644
--- a/docker/thirdparties/docker-health.sh
+++ b/docker/thirdparties/docker-health.sh
@@ -18,12 +18,12 @@
 
 docker_container_state_cmd() {
     local container_name="$1"
-    sudo docker inspect --format '{{.State.Status}}|{{if 
.State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "${container_name}"
+    docker_cli inspect --format '{{.State.Status}}|{{if 
.State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "${container_name}"
 }
 
 docker_container_env_cmd() {
     local container_name="$1"
-    sudo docker inspect --format '{{range .Config.Env}}{{println .}}{{end}}' 
"${container_name}"
+    docker_cli inspect --format '{{range .Config.Env}}{{println .}}{{end}}' 
"${container_name}"
 }
 
 docker_container_ready() {
diff --git a/docker/thirdparties/run-thirdparties-docker.sh 
b/docker/thirdparties/run-thirdparties-docker.sh
index 14babf73026..6402e4a379d 100755
--- a/docker/thirdparties/run-thirdparties-docker.sh
+++ b/docker/thirdparties/run-thirdparties-docker.sh
@@ -23,6 +23,13 @@
 set -eo pipefail
 
 ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
+HOST_OS="$(uname -s)"
+
+. "${ROOT}/docker-command.sh"
+
+docker_cli() {
+    docker "$@"
+}
 
 . "${ROOT}/custom_settings.env"
 . "${ROOT}/juicefs-helpers.sh"
@@ -38,7 +45,7 @@ Usage: $0 <options>
      -c mysql           start MySQL
      -c mysql,hive3     start MySQL and Hive3
      --stop             stop the specified components
-     --reserve-ports    reserve host ports by setting 
'net.ipv4.ip_local_reserved_ports' to avoid port already bind error
+     --reserve-ports    unsupported; configure host port reservations 
separately
      --no-load-data     do not load data into the components
      --load-parallel <num>  set the parallel number to load data, default is 
the 50% of CPU cores
      --hive-mode <mode>     hive startup mode: fast, refresh, rebuild
@@ -50,7 +57,7 @@ Usage: $0 <options>
     exit 1
 }
 
DEFAULT_COMPONENTS="mysql,es,hive2,hive3,pg,oracle,sqlserver,clickhouse,mariadb,iceberg,hudi,db2,oceanbase,kerberos,minio"
-ALL_COMPONENTS="${DEFAULT_COMPONENTS},kafka,lakesoul,ranger,polaris"
+ALL_COMPONENTS="${DEFAULT_COMPONENTS},iceberg-rest,kafka,lakesoul,ranger,polaris"
 COMPONENTS=$2
 HELP=0
 STOP=0
@@ -168,8 +175,10 @@ if [[ "${CONTAINER_UID}"x == "doris--"x ]]; then
     exit 1
 fi
 
+configure_docker_access
+
 if [[ -z "${HIVE_HOST_ALIAS:-}" ]]; then
-    export HIVE_HOST_ALIAS="hadoop-master-${HIVE_SHARED_ID}"
+    export HIVE_HOST_ALIAS=localhost
 fi
 
 echo "Components are: ${COMPONENTS}"
@@ -326,10 +335,8 @@ reserve_ports() {
         return
     fi
 
-    if [[ "${RESERVED_PORTS}"x != ""x ]]; then
-        echo "Reserve ports: ${RESERVED_PORTS}"
-        sudo sysctl -w net.ipv4.ip_local_reserved_ports="${RESERVED_PORTS}"
-    fi
+    echo "--reserve-ports changes host sysctl settings; configure it 
separately as an operator" >&2
+    return 1
 }
 
 JFS_META_FORMATTED=0
@@ -435,9 +442,9 @@ ensure_juicefs_meta_database() {
             return 0
         fi
 
-        mysql_container=$(sudo docker ps --format '{{.Names}}' | grep -E 
"(^|-)${CONTAINER_UID}mysql_57(-[0-9]+)?$" | head -n 1 || true)
+        mysql_container=$(docker_cli ps --format '{{.Names}}' | grep -E 
"(^|-)${CONTAINER_UID}mysql_57(-[0-9]+)?$" | head -n 1 || true)
         if [[ -n "${mysql_container}" ]]; then
-            if sudo docker exec "${mysql_container}" \
+            if docker_cli exec "${mysql_container}" \
                 mysql -uroot -p123456 -e "CREATE DATABASE IF NOT EXISTS 
\`${meta_db}\`;" >/dev/null 2>&1; then
                 return 0
             fi
@@ -462,16 +469,16 @@ ensure_juicefs_meta_database() {
         )
 
         for pg_container in "${pg_candidates[@]}"; do
-            if ! sudo docker ps --format '{{.Names}}' | grep -Fxq 
"${pg_container}"; then
+            if ! docker_cli ps --format '{{.Names}}' | grep -Fxq 
"${pg_container}"; then
                 continue
             fi
 
-            if sudo docker exec "${pg_container}" \
+            if docker_cli exec "${pg_container}" \
                 psql -U postgres -d postgres -tAc "SELECT 1 FROM pg_database 
WHERE datname='${meta_db}'" | grep -q '^1$'; then
                 return 0
             fi
 
-            if sudo docker exec "${pg_container}" \
+            if docker_cli exec "${pg_container}" \
                 psql -U postgres -d postgres -c "CREATE DATABASE 
\"${meta_db}\";" >/dev/null 2>&1; then
                 return 0
             fi
@@ -551,8 +558,8 @@ prepare_juicefs_meta_for_hive() {
     fi
 
     local bucket_dir="${JFS_BUCKET_DIR:-/tmp/jfs-bucket}"
-    sudo mkdir -p "${bucket_dir}"
-    sudo chmod 777 "${bucket_dir}"
+    mkdir -p "${bucket_dir}"
+    chmod_fixture_dir 777 "${bucket_dir}"
 
     # For local docker metadata DSNs (mysql/postgresql), ensure metadata 
database exists.
     ensure_juicefs_meta_database "${jfs_meta}"
@@ -568,7 +575,8 @@ prepare_juicefs_meta_for_hive() {
     # and will cause "juicefs format" to fail with "Storage ... is not empty".
     if [[ -d "${bucket_dir}" ]]; then
         echo "Cleaning stale JuiceFS bucket directory: ${bucket_dir}"
-        sudo rm -rf "${bucket_dir:?}"/*
+        reset_data_dirs "${bucket_dir:?}"
+        chmod_fixture_dir 777 "${bucket_dir}"
     fi
 
     if ! run_juicefs_cli \
@@ -597,9 +605,9 @@ compose_cmd() {
     shift 2
 
     if [[ -n "${env_file}" ]]; then
-        sudo docker compose -f "${compose_file}" --env-file "${env_file}" "$@"
+        docker_cli compose -f "${compose_file}" --env-file "${env_file}" "$@"
     else
-        sudo docker compose -f "${compose_file}" "$@"
+        docker_cli compose -f "${compose_file}" "$@"
     fi
 }
 
@@ -623,8 +631,7 @@ reset_data_dirs() {
     local data_dir
 
     for data_dir in "$@"; do
-        sudo mkdir -p "${data_dir}"
-        sudo rm -rf "${data_dir:?}"/*
+        reset_rootless_data_dir "${data_dir}"
     done
 }
 
@@ -833,7 +840,7 @@ dump_start_failure() {
     fi
 
     echo "===== unhealthy containers =====" >&2
-    sudo docker ps -a --filter 'health=unhealthy' --format '{{.Names}} | 
{{.Image}} | {{.Status}}' >&2 || true
+    docker_cli ps -a --filter 'health=unhealthy' --format '{{.Names}} | 
{{.Image}} | {{.Status}}' >&2 || true
 }
 
 print_started_summary() {
@@ -862,7 +869,7 @@ print_started_summary() {
             echo "  containers:"
             while read -r container_id; do
                 [[ -n "${container_id}" ]] || continue
-                sudo docker inspect --format '{{.Name}} | {{.Config.Image}} | 
{{.State.Status}} | health={{if 
.State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "${container_id}" \
+                docker_cli inspect --format '{{.Name}} | {{.Config.Image}} | 
{{.State.Status}} | health={{if 
.State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "${container_id}" \
                     2>/dev/null || true
             done <<<"${compose_ids}"
         fi
@@ -912,22 +919,16 @@ start_es() {
     register_stack_metadata "es" 
"${ROOT}/docker-compose/elasticsearch/es.yaml" 
"${ROOT}/docker-compose/elasticsearch/es.env"
     compose_down_stack "${ROOT}/docker-compose/elasticsearch/es.yaml" 
"${ROOT}/docker-compose/elasticsearch/es.env" --remove-orphans
     if [[ "${STOP}" -ne 1 ]]; then
-        sudo mkdir -p "${ROOT}"/docker-compose/elasticsearch/data/es6/
-        sudo rm -rf "${ROOT}"/docker-compose/elasticsearch/data/es6/*
-        sudo mkdir -p "${ROOT}"/docker-compose/elasticsearch/data/es7/
-        sudo rm -rf "${ROOT}"/docker-compose/elasticsearch/data/es7/*
-        sudo mkdir -p "${ROOT}"/docker-compose/elasticsearch/data/es8/
-        sudo rm -rf "${ROOT}"/docker-compose/elasticsearch/data/es8/*
-        sudo chmod -R 777 "${ROOT}"/docker-compose/elasticsearch/data
-        sudo mkdir -p "${ROOT}"/docker-compose/elasticsearch/logs/es6/
-        sudo rm -rf "${ROOT}"/docker-compose/elasticsearch/logs/es6/*
-        sudo mkdir -p "${ROOT}"/docker-compose/elasticsearch/logs/es7/
-        sudo rm -rf "${ROOT}"/docker-compose/elasticsearch/logs/es7/*
-        sudo mkdir -p "${ROOT}"/docker-compose/elasticsearch/logs/es8/
-        sudo rm -rf "${ROOT}"/docker-compose/elasticsearch/logs/es8/*
-        sudo chmod -R 777 "${ROOT}"/docker-compose/elasticsearch/logs
-        sudo chmod -R 777 "${ROOT}"/docker-compose/elasticsearch/config
-        compose_cmd "${ROOT}/docker-compose/elasticsearch/es.yaml" 
"${ROOT}/docker-compose/elasticsearch/es.env" up -d --remove-orphans
+        local version
+        for version in 5 6 7 8; do
+            reset_data_dirs \
+                "${ROOT}/docker-compose/elasticsearch/data/es${version}" \
+                "${ROOT}/docker-compose/elasticsearch/logs/es${version}"
+        done
+        chmod_fixture_dir 777 "${ROOT}/docker-compose/elasticsearch/data"
+        chmod_fixture_dir 777 "${ROOT}/docker-compose/elasticsearch/logs"
+        chmod_fixture_dir 777 "${ROOT}/docker-compose/elasticsearch/config"
+        compose_cmd "${ROOT}/docker-compose/elasticsearch/es.yaml" 
"${ROOT}/docker-compose/elasticsearch/es.env" up -d --wait --remove-orphans
     fi
 }
 
@@ -1017,7 +1018,7 @@ start_kafka() {
 
         for topic in "${topics[@]}"; do
             echo "Creating kafka topic '${topic}' for ${container_id}"
-            sudo docker exec "${container_id}" bash -c 
"/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server 
'${ip_host}:19193' --topic '${topic}'"
+            docker_cli exec "${container_id}" bash -c 
"/opt/bitnami/kafka/bin/kafka-topics.sh --create --bootstrap-server 
'${ip_host}:19193' --topic '${topic}'"
         done
 
     }
@@ -1028,7 +1029,7 @@ start_kafka() {
         local attempt
 
         for attempt in {1..30}; do
-            if sudo docker exec "${container_id}" bash -c 
"/opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server 
'${ip_host}:19193'" >/dev/null 2>&1; then
+            if docker_cli exec "${container_id}" bash -c 
"/opt/bitnami/kafka/bin/kafka-topics.sh --list --bootstrap-server 
'${ip_host}:19193'" >/dev/null 2>&1; then
                 return 0
             fi
             sleep 2
@@ -1039,7 +1040,7 @@ start_kafka() {
     }
 
     if [[ "${STOP}" -ne 1 ]]; then
-        compose_up_stack "${ROOT}/docker-compose/kafka/kafka.yaml" 
"${ROOT}/docker-compose/kafka/kafka.env" --build --remove-orphans -d
+        compose_up_stack "${ROOT}/docker-compose/kafka/kafka.yaml" 
"${ROOT}/docker-compose/kafka/kafka.env" --build --remove-orphans -d --wait
         wait_for_kafka_ready "${KAFKA_CONTAINER_ID}" "${IP_HOST}"
         create_kafka_topics "${KAFKA_CONTAINER_ID}" "${IP_HOST}"
     fi
@@ -1070,8 +1071,8 @@ ensure_hive_volumes() {
     local prefix="$1"
     local suffix
     for suffix in "${HIVE_VOLUME_SUFFIXES[@]}"; do
-        if ! sudo docker volume inspect "${prefix}-${suffix}" >/dev/null 2>&1; 
then
-            sudo docker volume create "${prefix}-${suffix}" >/dev/null
+        if ! docker_cli volume inspect "${prefix}-${suffix}" >/dev/null 2>&1; 
then
+            docker_cli volume create "${prefix}-${suffix}" >/dev/null
         fi
     done
 }
@@ -1080,13 +1081,13 @@ reset_hive_volumes() {
     local prefix="$1"
     local suffix
     for suffix in "${HIVE_VOLUME_SUFFIXES[@]}"; do
-        sudo docker volume rm -f "${prefix}-${suffix}" >/dev/null 2>&1 || true
+        docker_cli volume rm -f "${prefix}-${suffix}" >/dev/null 2>&1 || true
     done
 }
 
 hive_volume_is_populated() {
     local prefix="$1"
-    sudo docker run --rm \
+    docker_cli run --rm \
         -v "${prefix}-namenode:/vol:ro" \
         alpine test -f /vol/current/VERSION 2>/dev/null
 }
@@ -1186,7 +1187,7 @@ maybe_restore_baseline_to_volumes() {
     echo "[baseline] restoring volumes from extracted baseline cache..."
     local _t0
     _t0=$(date +%s)
-    sudo docker run --rm \
+    docker_cli run --rm \
         -v "${extracted_dir}:/baseline:ro" \
         -v "${prefix}-namenode:/restore/namenode" \
         -v "${prefix}-datanode:/restore/datanode" \
@@ -1230,32 +1231,14 @@ hive_metastore_container_for() {
 ensure_hosts_alias() {
     local alias_name="$1"
     local alias_ip="$2"
-    local tmp_hosts
-    local sudo_cmd=()
-
-    if [[ "$(id -u)" -ne 0 ]]; then
-        sudo_cmd=(sudo)
-    fi
-
-    tmp_hosts="$(mktemp)"
-    "${sudo_cmd[@]}" chmod a+w /etc/hosts
-    awk -v alias_name="${alias_name}" '
-        {
-            keep = 1
-            for (i = 2; i <= NF; ++i) {
-                if ($i == alias_name) {
-                    keep = 0
-                    break
-                }
-            }
-            if (keep) {
-                print
-            }
-        }
-    ' /etc/hosts >"${tmp_hosts}"
-    printf "%s %s\n" "${alias_ip}" "${alias_name}" >>"${tmp_hosts}"
-    "${sudo_cmd[@]}" cp "${tmp_hosts}" /etc/hosts
-    rm -f "${tmp_hosts}"
+    if [[ "${alias_name}" == "localhost" && "${alias_ip}" == "127.0.0.1" ]]; 
then
+        return 0
+    fi
+    if getent ahostsv4 "${alias_name}" | awk -v ip="${alias_ip}" '$1 == ip { 
found = 1 } END { exit !found }'; then
+        return 0
+    fi
+    echo "${alias_name} must already resolve to ${alias_ip}; set 
HIVE_HOST_ALIAS=localhost and IP_HOST=127.0.0.1 for local fixtures" >&2
+    return 1
 }
 
 render_hive_compose() {
@@ -1277,7 +1260,7 @@ render_hive_compose() {
 
 hive_compose_cmd() {
     local hive_version="$1"
-    sudo docker compose -p "${CONTAINER_UID}${hive_version}" -f 
"$(hive_compose_file_for "${hive_version}")" --env-file "$(hive_env_file_for 
"${hive_version}")" "${@:2}"
+    docker_cli compose -p "${CONTAINER_UID}${hive_version}" -f 
"$(hive_compose_file_for "${hive_version}")" --env-file "$(hive_env_file_for 
"${hive_version}")" "${@:2}"
 }
 
 exec_hive_script() {
@@ -1289,7 +1272,7 @@ exec_hive_script() {
     # -i: forward SIGINT/SIGTERM into container so Ctrl+C kills the 
in-container script
     #     instead of leaving an orphan that keeps mutating state.
     # stdbuf -oL -eL: line-buffer output so progress reaches the host log in 
real time.
-    sudo docker exec -i \
+    docker_cli exec -i \
         -e HIVE_BOOTSTRAP_GROUPS="${HIVE_BOOTSTRAP_GROUPS}" \
         -e LOAD_PARALLEL="${LOAD_PARALLEL}" \
         -e HIVE_HQL_PARALLEL="${HIVE_HQL_PARALLEL}" \
@@ -1334,6 +1317,12 @@ start_hive_stack() {
     local volume_prefix
     local baseline_restore_result="missing"
 
+    # localhost metadata and host-network services must use loopback on both
+    # sides, even when other components advertise the host's LAN address.
+    if [[ "${HIVE_HOST_ALIAS}" == "localhost" ]]; then
+        export IP_HOST=127.0.0.1
+    fi
+
     export HIVE_BOOTSTRAP_GROUPS="$(hive_bootstrap_groups_for 
"${hive_version}")"
     echo "${hive_version} selected bootstrap files: ${HIVE_BOOTSTRAP_GROUPS}"
 
@@ -1342,16 +1331,17 @@ start_hive_stack() {
     export HIVE_VOLUME_PREFIX="${volume_prefix}"
     log_hive_volumes "${hive_version}" "${volume_prefix}"
 
-    # Keep a stable hostname in metastore/HDFS metadata while allowing the
-    # backing host IP to change across restarts.
-    ensure_hosts_alias "${HIVE_HOST_ALIAS}" "${IP_HOST}"
-
     if [[ "${STOP}" -eq 1 ]]; then
         render_hive_compose "${hive_version}"
         hive_compose_cmd "${hive_version}" down
         return 0
     fi
 
+    # Keep a stable hostname in metastore/HDFS metadata while allowing the
+    # backing host IP to change across restarts. Stopping needs no DNS setup.
+    require_rootless_host_network
+    ensure_hosts_alias "${HIVE_HOST_ALIAS}" "${IP_HOST}"
+
     # refresh/rebuild: tear down the stack and clear volumes first.
     # fast: keep existing volumes and only restore the baseline when they are 
empty.
     if [[ "${HIVE_MODE}" == "rebuild" || "${HIVE_MODE}" == "refresh" ]]; then
@@ -1406,9 +1396,9 @@ start_iceberg() {
                 cd "${ICEBERG_DIR}" || exit 1
                 rm -f iceberg_data*.zip
                 wget -P "${ROOT}/docker-compose/iceberg" 
"https://${s3BucketName}.${s3Endpoint}/regression/datalake/pipeline_data/iceberg_data_spark40_paimon142.zip";
-                sudo unzip iceberg_data_spark40_paimon142.zip
-                sudo mv iceberg_data data
-                sudo rm -rf iceberg_data_spark40_paimon142.zip
+                unzip iceberg_data_spark40_paimon142.zip
+                mv iceberg_data data
+                rm -rf iceberg_data_spark40_paimon142.zip
             )
         else
             echo "${ICEBERG_DIR}/data exists, continue !"
@@ -1416,9 +1406,9 @@ start_iceberg() {
 
         if [[ ! -s "${LANCE_SPARK_JAR_PATH}" ]]; then
             echo "Downloading ${LANCE_SPARK_JAR}"
-            sudo mkdir -p "$(dirname "${LANCE_SPARK_JAR_PATH}")"
-            sudo wget -O "${LANCE_SPARK_JAR_PATH}.tmp" "${LANCE_SPARK_JAR_URL}"
-            sudo mv "${LANCE_SPARK_JAR_PATH}.tmp" "${LANCE_SPARK_JAR_PATH}"
+            mkdir -p "$(dirname "${LANCE_SPARK_JAR_PATH}")"
+            wget -O "${LANCE_SPARK_JAR_PATH}.tmp" "${LANCE_SPARK_JAR_URL}"
+            mv "${LANCE_SPARK_JAR_PATH}.tmp" "${LANCE_SPARK_JAR_PATH}"
         fi
 
         compose_up_stack "${ROOT}/docker-compose/iceberg/iceberg.yaml" 
"${ROOT}/docker-compose/iceberg/iceberg.env" -d --wait
@@ -1437,7 +1427,7 @@ start_hudi() {
     . "${HUDI_DIR}"/hudi.env
     set +a
     envsubst <"${HUDI_DIR}"/hudi.yaml.tpl >"${HUDI_DIR}"/hudi.yaml
-    sudo chmod +x "${HUDI_DIR}"/scripts/init.sh
+    chmod +x "${HUDI_DIR}"/scripts/init.sh
     register_stack_metadata "hudi" "${HUDI_DIR}/hudi.yaml" 
"${HUDI_DIR}/hudi.env"
     compose_down_stack "${HUDI_DIR}/hudi.yaml" "${HUDI_DIR}/hudi.env" 
--remove-orphans
     if [[ "${STOP}" -ne 1 ]]; then
@@ -1461,7 +1451,8 @@ start_lakesoul() {
     sed -i "s/doris--/${CONTAINER_UID}/g" 
"${ROOT}"/docker-compose/lakesoul/lakesoul.yaml
     register_stack_metadata "lakesoul" 
"${ROOT}/docker-compose/lakesoul/lakesoul.yaml" ""
     compose_cmd "${ROOT}/docker-compose/lakesoul/lakesoul.yaml" "" down 
--remove-orphans
-    sudo rm -rf "${ROOT}"/docker-compose/lakesoul/data
+    reset_data_dirs "${ROOT}/docker-compose/lakesoul/data"
+    rmdir "${ROOT}/docker-compose/lakesoul/data"
     if [[ "${STOP}" -ne 1 ]]; then
         echo "PREPARE_LAKESOUL_DATA"
         compose_cmd "${ROOT}/docker-compose/lakesoul/lakesoul.yaml" "" up -d
@@ -1496,18 +1487,18 @@ dump_kerberos_container_state() {
     local container="$1"
 
     echo "===== ${container} state =====" >&2
-    sudo docker inspect --format \
+    docker_cli inspect --format \
         'status={{.State.Status}} running={{.State.Running}} 
exitCode={{.State.ExitCode}} oomKilled={{.State.OOMKilled}} 
error={{.State.Error}}' \
         "${container}" >&2 || true
     echo "===== ${container} logs (tail -200) =====" >&2
-    sudo docker logs --tail 200 "${container}" >&2 || true
+    docker_cli logs --tail 200 "${container}" >&2 || true
 }
 
 cleanup_kerberos_ready_wait() {
     local wait_pid="$1"
 
     [[ -n "${wait_pid}" ]] || return 0
-    sudo kill -TERM -- "-${wait_pid}" >/dev/null 2>&1 || true
+    kill -TERM -- "-${wait_pid}" >/dev/null 2>&1 || true
     wait "${wait_pid}" >/dev/null 2>&1 || true
 }
 
@@ -1517,7 +1508,7 @@ wait_for_kerberos_ready() {
     local wait_status=0
 
     echo "Waiting for ${container} readiness event"
-    setsid sudo timeout --signal=TERM --kill-after=5s 20m bash -c '
+    setsid timeout --signal=TERM --kill-after=5s 20m bash -c '
         log_dir=$(mktemp -d)
         mkfifo "${log_dir}/container.log"
         docker logs --follow "$1" >"${log_dir}/container.log" 2>&1 &
@@ -1567,7 +1558,7 @@ validate_kerberos_container() {
     local container="$1"
 
     echo "Running one-shot readiness validation for ${container}"
-    if ! sudo docker exec "${container}" /opt/doris/health.sh; then
+    if ! docker_cli exec "${container}" /opt/doris/health.sh; then
         echo "ERROR: one-shot readiness validation failed for ${container}" >&2
         dump_kerberos_container_state "${container}"
         return 1
@@ -1642,21 +1633,12 @@ start_kerberos() {
             envsubst <"${KERBEROS_DIR}/conf/${config}.tpl" 
>"${KERBEROS_DIR}/conf/kerberos${i}/${config}"
         done
     done
-    sudo chmod a+w /etc/hosts
-    if ! awk -v ip="${IP_HOST}" '$1 == ip && $2 == "hadoop-master" { found = 1 
} END { exit !found }' /etc/hosts; then
-        sudo sed -i "1i${IP_HOST} hadoop-master" /etc/hosts
-    fi
-    if ! awk -v ip="${IP_HOST}" '$1 == ip && $2 == "hadoop-master-2" { found = 
1 } END { exit !found }' /etc/hosts; then
-        sudo sed -i "1i${IP_HOST} hadoop-master-2" /etc/hosts
-    fi
     register_stack_metadata "kerberos" "${KERBEROS_DIR}/kerberos.yaml" ""
     compose_cmd "${KERBEROS_DIR}/kerberos.yaml" "" down --remove-orphans
-    sudo rm -rf "${KERBEROS_DIR}/data"
     if [[ "${STOP}" -ne 1 ]]; then
+        require_rootless_host_network
         echo "PREPARE KERBEROS DATA"
-        rm -rf "${KERBEROS_DIR}"/two-kerberos-hives/*.keytab
-        rm -rf "${KERBEROS_DIR}"/two-kerberos-hives/*.jks
-        rm -rf "${KERBEROS_DIR}"/two-kerberos-hives/*.conf
+        reset_data_dirs "${KERBEROS_DIR}/data" 
"${KERBEROS_DIR}/two-kerberos-hives"
         compose_cmd "${KERBEROS_DIR}/kerberos.yaml" "" up --build 
--remove-orphans -d
         trap 'cleanup_kerberos_readiness_jobs "${readiness_pids[@]}"' EXIT
         trap 'exit 143' TERM
@@ -1678,9 +1660,13 @@ start_kerberos() {
         for container in "${containers[@]}"; do
             validate_kerberos_container "${container}"
         done
-        sudo ln -sfn "${KERBEROS_DIR}/two-kerberos-hives" /keytabs
-        sudo cp "${KERBEROS_DIR}/common/conf/doris-krb5.conf" 
/keytabs/krb5.conf
-        sudo cp "${KERBEROS_DIR}/common/conf/doris-krb5.conf" /etc/krb5.conf
+        sed -e "s/kdc = hadoop-master:/kdc = ${IP_HOST}:/" \
+            -e "s/kdc = hadoop-master-2:/kdc = ${IP_HOST}:/" \
+            -e "s#FILE:/var/log/#FILE:${KERBEROS_DIR}/two-kerberos-hives/#g" \
+            "${KERBEROS_DIR}/common/conf/doris-krb5.conf" \
+            >"${KERBEROS_DIR}/two-kerberos-hives/krb5.conf"
+        echo "Kerberos client files: ${KERBEROS_DIR}/two-kerberos-hives"
+        echo "For host clients: export 
KRB5_CONFIG=${KERBEROS_DIR}/two-kerberos-hives/krb5.conf"
         sleep 2
     fi
 }
@@ -1709,11 +1695,8 @@ start_polaris() {
     register_stack_metadata "polaris" "${POLARIS_DIR}/docker-compose.yaml" ""
     compose_cmd "${POLARIS_DIR}/docker-compose.yaml" "" down --remove-orphans
     if [[ "${STOP}" -ne 1 ]]; then
-        # polaris-init is a one-shot service. Keep it out of `up --wait`,
-        # otherwise Compose reports the successfully exited container as a
-        # failed stack startup.
-        compose_cmd "${POLARIS_DIR}/docker-compose.yaml" "" up -d --wait 
--remove-orphans polaris
-        compose_cmd "${POLARIS_DIR}/docker-compose.yaml" "" run --rm --no-deps 
polaris-init
+        # polaris-init stays alive after initialization; its healthcheck marks 
readiness.
+        compose_cmd "${POLARIS_DIR}/docker-compose.yaml" "" up -d --wait 
--remove-orphans
     fi
 }
 
@@ -1740,10 +1723,22 @@ start_iceberg_rest() {
     export CONTAINER_UID=${CONTAINER_UID}
     . "${ROOT}"/docker-compose/iceberg-rest/iceberg-rest_settings.env
     envsubst <"${ICEBERG_REST_DIR}/docker-compose.yaml.tpl" 
>"${ICEBERG_REST_DIR}/docker-compose.yaml"
+    if [[ "${HOST_OS}" != "Darwin" ]]; then
+        if [[ "${STOP}" -ne 1 ]]; then
+            require_rootless_host_network
+        fi
+        # A host HDFS client must reach the DataNode address returned by the
+        # NameNode. A rootless bridge address is not routable from the host.
+        envsubst <"${ICEBERG_REST_DIR}/hdfs-rootless.yaml.tpl" 
>"${ICEBERG_REST_DIR}/hdfs-rootless.yaml"
+        docker_cli compose -f "${ICEBERG_REST_DIR}/docker-compose.yaml" \
+            -f "${ICEBERG_REST_DIR}/hdfs-rootless.yaml" config \
+            >"${ICEBERG_REST_DIR}/docker-compose.rootless.yaml"
+        mv "${ICEBERG_REST_DIR}/docker-compose.rootless.yaml" 
"${ICEBERG_REST_DIR}/docker-compose.yaml"
+    fi
     register_stack_metadata "iceberg-rest" 
"${ICEBERG_REST_DIR}/docker-compose.yaml" ""
     compose_cmd "${ICEBERG_REST_DIR}/docker-compose.yaml" "" down 
--remove-orphans
     if [[ "${STOP}" -ne 1 ]]; then
-        # Start all three REST catalogs (S3, OSS, COS)
+        # Start the REST catalogs and their built-in HDFS.
         compose_cmd "${ICEBERG_REST_DIR}/docker-compose.yaml" "" up -d 
--remove-orphans --wait
     fi
 }
@@ -1754,7 +1749,7 @@ reserve_ports
 
 # Ensure hive data is downloaded before starting hive2/hive3, but only once
 need_prepare_hive_data=0
-if [[ "$NEED_LOAD_DATA" -eq 1 ]]; then
+if [[ "${STOP}" -ne 1 && "$NEED_LOAD_DATA" -eq 1 ]]; then
     if [[ "${RUN_HIVE2}" -eq 1 ]] || [[ "${RUN_HIVE3}" -eq 1 ]]; then
         if [[ "${HIVE_MODE}" == "refresh" || "${HIVE_MODE}" == "rebuild" ]]; 
then
             need_prepare_hive_data=1
diff --git a/docker/thirdparties/test/docker-command-test.sh 
b/docker/thirdparties/test/docker-command-test.sh
new file mode 100755
index 00000000000..3d7c0eab304
--- /dev/null
+++ b/docker/thirdparties/test/docker-command-test.sh
@@ -0,0 +1,138 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -eo pipefail
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." &>/dev/null && pwd)"
+TEST_DIR="$(mktemp -d /tmp/doris-docker-test.XXXXXX)"
+TEST_LOG="${TEST_DIR}/commands.log"
+ORIGINAL_PATH="${PATH}"
+trap 'rm -rf "${TEST_DIR}"' EXIT
+
+fail() {
+    echo "FAIL: $*" >&2
+    exit 1
+}
+
+mkdir -p "${TEST_DIR}/bin"
+
+cat >"${TEST_DIR}/bin/docker" <<'EOF'
+#!/usr/bin/env bash
+if [[ "${1:-}" == "info" ]]; then
+    if [[ "${DOCKER_INFO_EXIT:-0}" -ne 0 ]]; then
+        exit "${DOCKER_INFO_EXIT}"
+    fi
+    if [[ "${2:-}" == "--format" ]]; then
+        if [[ "${3}" == '{{.ServerVersion}}' ]]; then
+            printf '%s\n' "${DOCKER_SERVER_VERSION}"
+        else
+            printf '["name=%s"]\n' "${DOCKER_SECURITY_MODE}"
+        fi
+    fi
+    exit 0
+fi
+printf 'docker %s\n' "$*" >>"${TEST_LOG}"
+EOF
+
+cat >"${TEST_DIR}/bin/sudo" <<'EOF'
+#!/usr/bin/env bash
+printf 'sudo %s\n' "$*" >>"${TEST_LOG}"
+exit 125
+EOF
+
+chmod +x "${TEST_DIR}/bin/docker" "${TEST_DIR}/bin/sudo"
+export PATH="${TEST_DIR}/bin:${ORIGINAL_PATH}"
+export TEST_LOG DOCKER_SECURITY_MODE
+HOST_OS=Linux
+
+# shellcheck source=../docker-command.sh
+. "${ROOT}/docker-command.sh"
+
+docker_cli() {
+    docker "$@"
+}
+
+unset DOCKER_HOST DOCKER_CONTEXT
+DOCKER_SECURITY_MODE=rootless
+configure_docker_access
+: >"${TEST_LOG}"
+export DOCKER_USE_SUDO=1 # A legacy environment must not enable elevation.
+docker_cli ps
+unset DOCKER_USE_SUDO
+grep -Fxq 'docker ps' "${TEST_LOG}" || fail "docker_cli did not call Docker 
directly"
+if grep -q '^sudo ' "${TEST_LOG}"; then
+    fail "rootless mode invoked sudo"
+fi
+
+export DOCKER_SERVER_VERSION=29.4.0
+if require_rootless_host_network 2>/dev/null; then
+    fail "rootless Docker before 29.5 was accepted for host networking"
+fi
+DOCKER_SERVER_VERSION=29.5.0
+require_rootless_host_network || fail "rootless Docker 29.5 was rejected"
+DOCKER_SERVER_VERSION=30.0.0
+require_rootless_host_network || fail "newer rootless Docker was rejected"
+
+DOCKER_SECURITY_MODE=seccomp
+: >"${TEST_LOG}"
+if configure_docker_access 2>/dev/null; then
+    fail "Linux accepted a rootful daemon"
+fi
+[[ ! -s "${TEST_LOG}" ]] || fail "a rootful daemon triggered elevation"
+
+# macOS keeps direct Docker Desktop access without Linux rootless detection.
+HOST_OS=Darwin
+configure_docker_access || fail "Docker Desktop was rejected"
+require_rootless_host_network || fail "Linux version requirement applied to 
macOS"
+HOST_OS=Linux
+DOCKER_SECURITY_MODE=rootless
+
+# Discover the user's socket even if the default system daemon is accessible.
+export XDG_RUNTIME_DIR="${TEST_DIR}/runtime"
+mkdir -p "${XDG_RUNTIME_DIR}"
+python3 - "${XDG_RUNTIME_DIR}/docker.sock" <<'PY'
+import socket
+import sys
+connection = socket.socket(socket.AF_UNIX)
+connection.bind(sys.argv[1])
+connection.close()
+PY
+unset DOCKER_HOST DOCKER_CONTEXT
+configure_docker_access
+[[ "${DOCKER_HOST}" == "unix://${XDG_RUNTIME_DIR}/docker.sock" ]] || fail 
"user socket was not discovered"
+
+DOCKER_HOST=unix:///explicit/docker.sock
+configure_docker_access
+[[ "${DOCKER_HOST}" == unix:///explicit/docker.sock ]] || fail "explicit 
endpoint was replaced"
+unset DOCKER_HOST
+export DOCKER_CONTEXT=review-rootless
+configure_docker_access
+[[ -z "${DOCKER_HOST+x}" ]] || fail "explicit context was replaced"
+unset DOCKER_CONTEXT
+
+# A failed daemon probe must not run a host operation or silently escalate it.
+export DOCKER_INFO_EXIT=17
+export DOCKER_HOST="unix://${XDG_RUNTIME_DIR}/docker.sock"
+: >"${TEST_LOG}"
+status=0
+(configure_docker_access && touch "${TEST_DIR}/unexpected") 2>/dev/null || 
status=$?
+[[ "${status}" -eq 1 ]] || fail "failed daemon probe did not propagate its 
error"
+[[ ! -e "${TEST_DIR}/unexpected" ]] || fail "host command ran after daemon 
probe failed"
+[[ ! -s "${TEST_LOG}" ]] || fail "failed daemon probe invoked sudo or another 
Docker operation"
+
+echo "PASS"
diff --git a/docker/thirdparties/test/launcher-stop-test.sh 
b/docker/thirdparties/test/launcher-stop-test.sh
new file mode 100644
index 00000000000..c8b2b43150b
--- /dev/null
+++ b/docker/thirdparties/test/launcher-stop-test.sh
@@ -0,0 +1,79 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Run the real CLI against an isolated fixture tree and a Docker stub.
+# Requires Bash 4+, like the launcher; does not access a real daemon.
+set -euo pipefail
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." &>/dev/null && pwd)"
+test_dir="$(mktemp -d /tmp/doris-stop-test.XXXXXX)"
+trap 'rm -rf "${test_dir}"' EXIT
+fixture="${test_dir}/thirdparties"
+mkdir -p "${fixture}" "${test_dir}/bin"
+for file in \
+    run-thirdparties-docker.sh docker-command.sh docker-health.sh \
+    juicefs-helpers.sh custom_settings.env \
+    docker-compose/hive/scripts/bootstrap/bootstrap-groups.sh \
+    docker-compose/hive/hive-3x_settings.env \
+    docker-compose/hive/hadoop-hive.env.tpl \
+    docker-compose/hive/hadoop-hive-3x.env.tpl \
+    docker-compose/hive/hive-3x.yaml.tpl; do
+    mkdir -p "$(dirname "${fixture}/${file}")"
+    cp "${ROOT}/${file}" "${fixture}/${file}"
+done
+# This branch's custom_settings.env sets CONTAINER_UID unconditionally; 
rewrite it the way CI does.
+sed 's/^CONTAINER_UID="doris--"/CONTAINER_UID="rootless-stop-test-"/' \
+    "${ROOT}/custom_settings.env" >"${fixture}/custom_settings.env"
+
+export STOP_TEST_DOCKER_LOG="${test_dir}/docker.log"
+export STOP_TEST_PREPARE_MARKER="${test_dir}/prepare-called"
+cat >"${fixture}/docker-compose/hive/scripts/prepare-hive-data.sh" <<'EOF'
+touch "${STOP_TEST_PREPARE_MARKER}"
+exit 99
+EOF
+cat >"${test_dir}/bin/docker" <<'EOF'
+#!/usr/bin/env bash
+if [[ "${1:-}" == info ]]; then
+    echo '["name=rootless"]'
+    exit 0
+fi
+printf '%s\n' "$*" >>"${STOP_TEST_DOCKER_LOG}"
+[[ "${1:-}" == compose ]] || exit 1
+for argument in "$@"; do
+    if [[ "${argument}" == down ]]; then exit 0; fi
+done
+exit 1
+EOF
+cat >"${test_dir}/bin/envsubst" <<'EOF'
+#!/bin/sh
+cat
+EOF
+chmod +x "${test_dir}/bin/docker" "${test_dir}/bin/envsubst"
+
+export PATH="${test_dir}/bin:${PATH}"
+export CONTAINER_UID=rootless-stop-test-
+export DOCKER_HOST=unix:///rootless-stop-test.sock
+export IP_HOST=127.0.0.1
+export HIVE_HOST_ALIAS=localhost
+export JFS_CLUSTER_META=redis://127.0.0.1:6379/0
+export HIVE_MODE=refresh
+bash "${fixture}/run-thirdparties-docker.sh" -c hive3 --stop \
+    >"${test_dir}/launcher.log" 2>&1
+[[ ! -e "${STOP_TEST_PREPARE_MARKER}" ]]
+grep -q 'compose -p rootless-stop-test-hive3 ' "${STOP_TEST_DOCKER_LOG}"
+grep -q ' down$' "${STOP_TEST_DOCKER_LOG}"
+echo "Hive stop bypasses data preparation and only tears down compose"
diff --git a/docker/thirdparties/test/snapshot-hive-baseline-test.sh 
b/docker/thirdparties/test/snapshot-hive-baseline-test.sh
new file mode 100644
index 00000000000..8c03a0eee43
--- /dev/null
+++ b/docker/thirdparties/test/snapshot-hive-baseline-test.sh
@@ -0,0 +1,99 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Integration test: requires the current user's Docker daemon and Python 3.
+set -euo pipefail
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." &>/dev/null && pwd)"
+HOST_OS="$(uname -s)"
+. "${ROOT}/docker-command.sh"
+configure_docker_access
+
+test_dir="$(mktemp -d /tmp/doris-snapshot-test.XXXXXX)"
+prefix="$(basename "${test_dir}" | tr '[:upper:]' '[:lower:]')"
+image="${THIRDPARTY_FILESYSTEM_IMAGE:-alpine:3.20}"
+snapshot="${ROOT}/docker-compose/hive/scripts/snapshot-hive-baseline.sh"
+volumes=()
+
+cleanup() {
+    if ((${#volumes[@]})); then
+        docker volume rm "${volumes[@]}" >/dev/null
+    fi
+    rm -rf "${test_dir}"
+}
+trap cleanup EXIT
+
+# Missing volumes must fail before producing an output file.
+if bash "${snapshot}" "${prefix}" "${test_dir}/missing.tar.gz" 
>"${test_dir}/missing.log" 2>&1; then
+    echo "FAIL: snapshot accepted missing volumes" >&2
+    exit 1
+fi
+[[ ! -e "${test_dir}/missing.tar.gz" ]]
+
+for suffix in namenode datanode pgdata state; do
+    volumes+=("${prefix}-${suffix}")
+    docker volume create "${prefix}-${suffix}" >/dev/null
+done
+
+docker run --rm --network none --security-opt no-new-privileges \
+    -v "${prefix}-namenode:/fixture/namenode" \
+    -v "${prefix}-datanode:/fixture/datanode" \
+    -v "${prefix}-pgdata:/fixture/pgdata" \
+    -v "${prefix}-state:/fixture/state" \
+    "${image}" sh -ec '
+        for part in namenode datanode pgdata state; do
+            printf "%s" "$part" >"/fixture/$part/payload"
+        done
+        printf legacy >/fixture/state/baseline.version
+        chown 123:456 /fixture/pgdata /fixture/pgdata/payload
+        chmod 700 /fixture/pgdata
+        chmod 600 /fixture/pgdata/payload
+    '
+
+bash "${snapshot}" "${prefix}" "${test_dir}/baseline.tar.gz"
+python3 - "${test_dir}/baseline.tar.gz" <<'PY'
+import sys
+import tarfile
+
+with tarfile.open(sys.argv[1]) as archive:
+    for part in ['namenode', 'datanode', 'pgdata', 'state']:
+        assert archive.extractfile(part + '/payload').read() == part.encode()
+    assert 'state/baseline.version' not in archive.getnames()
+    entry = archive.getmember('pgdata/payload')
+    assert (entry.uid, entry.gid, entry.mode) == (123, 456, 0o600)
+PY
+docker run --rm --network none -v "${prefix}-state:/fixture:ro" \
+    "${image}" test -f /fixture/baseline.version
+
+# An interrupted exporter must preserve an existing baseline and remove its
+# temporary output. Forward info/volume queries to the real rootless daemon.
+printf previous-baseline >"${test_dir}/previous.tar.gz"
+docker() {
+    if [[ "${1:-}" == "run" ]]; then
+        return 42
+    fi
+    command docker "$@"
+}
+export -f docker
+status=0
+bash "${snapshot}" "${prefix}" "${test_dir}/previous.tar.gz" \
+    >"${test_dir}/failure.log" 2>&1 || status=$?
+unset -f docker
+[[ "${status}" -eq 42 ]]
+[[ "$(cat "${test_dir}/previous.tar.gz")" == previous-baseline ]]
+[[ "$(find "${test_dir}" -name 'previous.tar.gz.*' | wc -l | tr -d ' ')" -eq 0 
]]
+echo "Snapshot export, UID/GID preservation and failure cleanup passed"


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to