imbajin commented on code in PR #3149: URL: https://github.com/apache/hugegraph/pull/3149#discussion_r3869121552
########## docker/hugegraph-hubble-3x3.non-auth.properties: ########## @@ -0,0 +1,51 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# Hubble configuration for the 3-node cluster running WITHOUT authentication, +# i.e. started with docker-compose-3x3.non-auth.yml. It is identical to +# hugegraph-hubble-3x3.properties except for auth.enabled below. Use it only +# on a trusted local network; the default stack keeps authentication on. +# +# This file REPLACES the image's /hubble/conf/hugegraph-hubble.properties +# wholesale: it is bind-mounted over it, not merged with it. Keys absent +# here therefore fall back to the code defaults in HubbleOptions, not to the +# values in the shipped conf file. Re-check against the shipped conf when +# upgrading Hubble. + +server.host=0.0.0.0 +server.port=8088 + +cluster=hg +idc=docker + +pd.enabled=true +# Hubble's code default is auth.enabled=true. Without this line the UI would +# demand a login against Servers that have no users configured. +auth.enabled=false Review Comment: ⚠️ The documented no-auth flow cannot disable Hubble login. Evidence: current apache/hugegraph-toolchain master still registers LoginInterceptor for /api/** and rejects requests without a session token; it has no auth.enabled option or consumer, while this file only adds auth.enabled=false. Starting with this properties file therefore still leaves Hubble's API behind its login, so the advertised no-auth flow cannot operate as documented. Implement a supported Hubble auth-disable setting or remove and document this unsupported Hubble no-auth path. ########## docker/README.md: ########## @@ -168,14 +175,170 @@ To validate local images without Compose replacing them with remote `latest`: ## 3-Node Cluster Quickstart +The cluster and the Hubble add-on share one named Docker network so Hubble +can attach to a running cluster without touching it. Treat that network as a +trust boundary: only the Server layer performs real authentication. Store +serves its control APIs with no credentials at all, and while PD's REST +control APIs do require an `Authorization` header, PD only checks that the +Basic-auth *user* is one of its internal service names and never validates +the password — so any client on the network can read and drive them. Note +that this stack depends on that behaviour: Hubble reaches PD as the service +name `hubble` with an empty password, so tightening PD's credential check +would also break Hubble's operations view. Because of that, PD and Store +control-plane ports and the three Server REST ports bind to `127.0.0.1` by +default, so an unauthenticated control plane is not reachable from outside the +host. Publish them more widely only by setting `HUGEGRAPH_CONTROL_PLANE_HOST` +or `HUGEGRAPH_SERVER_PUBLISH_HOST`, and only behind a network ACL or TLS +terminator you control. Any container on the host can still join the network +by declaring the well-known name. One-time setup: +write the required credentials to a mode-600 `docker/.env` and create the +network. +The cluster file requires both credentials — the admin password enables +authentication, and every Server replica must share one token secret so a +token issued by any server validates on all of them. The `:?` guards fire +on every Compose subcommand, including `down`. + +```bash +( + set -eu + cd docker + command -v openssl >/dev/null 2>&1 || { echo "openssl not found" >&2; exit 1; } + [ -e .env ] || install -m 600 /dev/null .env + chmod 600 .env + # Keep appends on their own lines even if the file was hand-edited. + [ ! -s .env ] || [ -z "$(tail -c1 .env)" ] || printf '\n' >> .env + pat='^[[:space:]]*(export[[:space:]]+)?' + if ! grep -Eq "${pat}HUGEGRAPH_ADMIN_PASSWORD=" .env; then + admin_password="$(openssl rand -base64 12)" + printf "HUGEGRAPH_ADMIN_PASSWORD='%s'\n" "${admin_password}" >> .env + unset admin_password + fi + if ! grep -Eq "${pat}HUGEGRAPH_AUTH_TOKEN_SECRET=" .env; then + token_secret="$(openssl rand -hex 32)" + printf "HUGEGRAPH_AUTH_TOKEN_SECRET='%s'\n" "${token_secret}" >> .env + unset token_secret + fi + # Parse values as data. Do not source .env as shell: a value such as + # HUGEGRAPH_ADMIN_PASSWORD=$(...) would run on the operator host. + # The optional export group is capture 1; the quoted value is capture 2. + read_dotenv_value() { + key="$1" + key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}=" + count="$(grep -Ec "${key_pattern}" .env || true)" + [ "${count}" -eq 1 ] || return 1 + value="$(sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env)" + # sed prints nothing and still exits 0 when the value is unquoted or + # double-quoted, which would otherwise hand back an empty credential. + [ -n "${value}" ] || return 1 + printf '%s\n' "${value}" + } + if ! admin_value="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)"; then + echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2 + exit 1 + fi + if ! token_value="$(read_dotenv_value HUGEGRAPH_AUTH_TOKEN_SECRET)"; then + echo "HUGEGRAPH_AUTH_TOKEN_SECRET must use the documented single-quoted format" >&2 + exit 1 + fi + [ -n "${admin_value}" ] || { + echo "HUGEGRAPH_ADMIN_PASSWORD must be non-empty in docker/.env" >&2 + exit 1 + } + [ "${#token_value}" -ge 32 ] || { + echo "HUGEGRAPH_AUTH_TOKEN_SECRET must be at least 32 bytes in docker/.env" >&2 + exit 1 + } + unset admin_value token_value + # The shared cluster network. To override the name, export + # HUGEGRAPH_NETWORK in this shell before running the block — a value + # in docker/.env is read by Compose, not by this script. + net="${HUGEGRAPH_NETWORK:-hugegraph-net}" + docker network inspect "${net}" >/dev/null 2>&1 || + docker network create "${net}" + # Hubble's volumes are external for the same reason the network is: both + # documented flows share them, and an external volume is not destroyed by a + # `down -v` from either one. + for vol in "${HUBBLE_DB_VOLUME:-hugegraph-hubble-db}" \ Review Comment: ⚠️ The quickstart volume setup ignores custom names stored in docker/.env. Evidence: this shell loop expands HUBBLE_DB_VOLUME and HUBBLE_UPLOAD_VOLUME from the process environment, but Compose later reads those variables from .env; setting either there leaves only the default volume created and combined up fails with an external-volume-not-found error. Export these variables before the block or parse the .env values here. ########## docker/README.md: ########## @@ -287,17 +759,28 @@ Configuration is injected via environment variables. The old `docker/configs/app > PD startup path uses the explicit `auth.admin_pa` value when it first creates > the administrator. Changing it later does not rotate an existing password. -The single-node Compose files also accept these deployment-level overrides: - -| Variable | Default | Description | -|----------|---------|-------------| -| `HUGEGRAPH_SERVER_IMAGE` | `hugegraph/server:<version>` | Complete Server image reference | -| `HUGEGRAPH_SERVER_PULL_POLICY` | `always` (`build` for dev) | Server pull policy | -| `HUBBLE_IMAGE` | `hugegraph/hubble:<version>` | Complete Hubble image reference | -| `HUBBLE_PULL_POLICY` | `always` (`missing` for dev) | Hubble pull policy | -| `HUBBLE_PUBLISH_HOST` | `127.0.0.1` | Hubble host bind address; remote access requires an HTTPS reverse proxy | -| `HUGEGRAPH_ADMIN_PASSWORD` | required (`docker/.env`) | Initial admin password; no public default is provided | -| `HUGEGRAPH_AUTH_TOKEN_SECRET` | generated | JWT signing secret; explicit values must be at least 32 bytes | +The Compose files also accept these deployment-level overrides; the +"Used by" column names the files that read each variable (single = the +single-node files, cluster = `docker-compose-3pd-3store-3server.yml`, +add-on = `docker-compose-hubble.yml`): + +| Variable | Used by | Default | Description | +|----------|---------|---------|-------------| +| `HUGEGRAPH_VERSION` | single (quickstart), cluster, add-on | `latest` | Shared image tag for PD, Store, Server, and Hubble; pin it in `docker/.env` so these files resolve the same release. The dev file builds from source and defaults Hubble to `latest`; set `HUBBLE_IMAGE` to pin it | +| `HUGEGRAPH_SERVER_IMAGE` | single | `hugegraph/server:<version>` | Complete Server image reference | +| `HUGEGRAPH_SERVER_PULL_POLICY` | single | `always` (`build` for dev) | Server pull policy | +| `HUBBLE_IMAGE` | single, add-on | `hugegraph/hubble:<version>` | Complete Hubble image reference | +| `HUBBLE_PULL_POLICY` | single, add-on | `always` (`missing` for dev) | Hubble pull policy | +| `HUBBLE_PUBLISH_HOST` | single, add-on | `127.0.0.1` | Hubble host bind address; remote access requires an HTTPS reverse proxy | +| `HUBBLE_PROPERTIES` | add-on | `./hugegraph-hubble-3x3.properties` | Hubble properties file bind-mounted into the container; set it to `./hugegraph-hubble-3x3.non-auth.properties` when the cluster runs without authentication | +| `HUBBLE_DB_VOLUME` | add-on | `hugegraph-hubble-db` | Volume holding Hubble's H2 database; the explicit name is what lets the attach and combined flows share state, so change it only to run a second independent Hubble | +| `HUBBLE_UPLOAD_VOLUME` | add-on | `hugegraph-hubble-upload-files` | Volume holding Hubble's uploaded files; same naming caveat as `HUBBLE_DB_VOLUME` | +| `HUGEGRAPH_NETWORK` | cluster, add-on | `hugegraph-net` | Pre-created external Docker network shared by the 3-node cluster and the Hubble add-on; the single-node files use their own project bridge instead | +| `HUGEGRAPH_PULL_POLICY` | cluster | `always` | Pull policy for the PD, Store, and Server images. The default refreshes a stale cached `latest`; set it to `missing` to run offline or against locally built tags | Review Comment: 🧹 `missing` does not guarantee offline use. Evidence: Docker Compose pulls an image when it is absent under `pull_policy: missing`, while this table says `HUGEGRAPH_PULL_POLICY=missing` runs offline; the same guide uses `never` for local images. Document `never` for strict offline or local use and reserve `missing` for pull-if-absent. ########## docker/README.md: ########## @@ -196,11 +359,320 @@ curl http://localhost:8520/v1/health # Check Server (Graph API) curl http://localhost:8080/versions -# List registered stores via PD -curl http://localhost:8620/v1/stores +# Every PD endpoint except /v1/health, /actuator/* and /v1/prom/targets/* +# needs an Authorization header. PD only checks that the Basic-auth user is +# one of its internal service names, so the empty password below is enough +# — and it grants the full PD control plane, writes included, not just these +# reads. That is exactly why the shared network must be treated as a trust +# boundary. Without the header PD answers with an exception body, not data. +pd_auth="Authorization: Basic $(printf 'hubble:' | base64)" + +# List registered stores via PD (expect three, each "state":"Up") +curl -H "${pd_auth}" http://localhost:8620/v1/stores # List partitions -curl http://localhost:8620/v1/partitions +curl -H "${pd_auth}" http://localhost:8620/v1/partitions +``` + +Confirm authentication actually engaged — `/versions` stays open by design, +so it cannot tell you whether auth is on. A graph read without credentials +must be rejected: + +```bash +cd docker +# Expect 401 on all three replicas +for port in 8080 8081 8082; do + curl -s -o /dev/null -w "${port}: %{http_code}\n" \ + "http://localhost:${port}/graphs/hugegraph/schema/vertexlabels"; +done + +# And a signed-in read must succeed. Parse the generated single-quoted +# value without executing docker/.env as shell. Passing the credential +# through --config keeps it out of argv, where `ps` would expose it. +# The subshell keeps the `exit 1` below from closing an interactive shell. +( +set -eu +read_dotenv_value() { + key="$1" + key_pattern="^[[:space:]]*(export[[:space:]]+)?${key}=" + count="$(grep -Ec "${key_pattern}" .env || true)" + [ "${count}" -eq 1 ] || return 1 + value="$(sed -nE "s/${key_pattern}'([^']*)'[[:space:]]*$/\\2/p" .env)" + # sed prints nothing and still exits 0 when the value is unquoted or + # double-quoted, which would otherwise hand back an empty credential. + [ -n "${value}" ] || return 1 + printf '%s\n' "${value}" +} +HUGEGRAPH_ADMIN_PASSWORD="$(read_dotenv_value HUGEGRAPH_ADMIN_PASSWORD)" || { + echo "HUGEGRAPH_ADMIN_PASSWORD must use the documented single-quoted format" >&2 + exit 1 +} +curl -s -o /dev/null -w '%{http_code}\n' \ + --config <(printf 'user = "admin:%s"\n' "${HUGEGRAPH_ADMIN_PASSWORD}") \ + http://localhost:8080/graphs/hugegraph/schema/vertexlabels +) +``` + +`200` from the second command with `401` from all three of the first means +authentication is on and working. If the first command returns `200`, the +running image ignored `PASSWORD` and the cluster is **unauthenticated** — +the most likely cause is an image older than the release this integration +needs (see the version note in the quickstart above). + +### Running without authentication + +The cluster above authenticates, and that is the right default: it is the +configuration most people copy, and an open graph database on a reachable +port is a bad surprise. For a throwaway local trial on a trusted network you +can opt out with `docker-compose-3x3.non-auth.yml`, which drops the admin +password and the JWT secret from all three Servers. Anyone who can reach the +published ports then has full read and write access to every graph, so do not +use it anywhere else. + +```bash +cd docker + +# The network and Hubble's volumes are external in every flow, so create them +# once even here. This mode skips only the credential half of the setup block. +docker network inspect hugegraph-net >/dev/null 2>&1 || + docker network create hugegraph-net +for vol in hugegraph-hubble-db hugegraph-hubble-upload-files; do + docker volume inspect "${vol}" >/dev/null 2>&1 || + docker volume create "${vol}" >/dev/null +done + +# No docker/.env is needed. The two placeholders exist only because Compose +# evaluates the base file's required-variable guards before applying the +# override, which then drops both variables, so neither value reaches a +# container. Needs Compose v2.24+. +# +# Keep them inline on every command; do not write them into docker/.env. They +# are published values rather than secrets, and the token placeholder is +# deliberately shorter than the 32 bytes the Server requires, so if it ever +# reaches an authenticated Server that Server refuses to start instead of +# signing tokens with a key anyone can read here. +HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \ +HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \ + docker compose -f docker-compose-3pd-3store-3server.yml \ + -f docker-compose-3x3.non-auth.yml up -d + +# Every graph request now succeeds without credentials: expect 200, not 401. +curl -s -o /dev/null -w '%{http_code}\n' \ + http://localhost:8080/graphs/hugegraph/schema/vertexlabels + +# Teardown. The base file's `:?` guards fire on every subcommand, `down` +# included, so the same two placeholders are required here. +HUGEGRAPH_ADMIN_PASSWORD=non-auth-placeholder \ +HUGEGRAPH_AUTH_TOKEN_SECRET=non-auth-placeholder \ + docker compose -f docker-compose-3pd-3store-3server.yml \ + -f docker-compose-3x3.non-auth.yml down -v +``` + +Hubble needs the matching configuration, because its own default is to demand +a login. Point it at the non-auth properties file when you attach it: + +```bash +HUBBLE_PROPERTIES=./hugegraph-hubble-3x3.non-auth.properties \ + docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d +``` + +`HUBBLE_PROPERTIES` is not remembered between commands. It selects a bind +mount, so any later Compose command for Hubble that omits it silently +recreates the container against the authenticated properties file, and Hubble +comes back demanding a login the Servers cannot serve. Repeat the variable on +every Hubble command in this mode, including `up -d --no-deps hubble` and the +local-image commands below. + +<details> +<summary>Prompt for an AI assistant</summary> + +Copy this to an assistant instead of transcribing the commands by hand. + +```markdown +Start the Apache HugeGraph 3-node Docker cluster (3 PD, 3 Store, 3 Server) +with Hubble, from the `docker/` directory of the hugegraph repository. + +Use the authenticated default unless I say otherwise: +1. Follow "3-Node Cluster Quickstart" in docker/README.md to create + docker/.env with a generated admin password and JWT token secret, and to + create the external hugegraph-net network. +2. Start the cluster with docker-compose-3pd-3store-3server.yml, then attach + Hubble with docker-compose-hubble.yml. +3. Verify: an unauthenticated graph request returns 401, an authenticated one + returns 200, and Hubble answers on http://localhost:8088. +4. Tell me the admin password so I can sign in to Hubble. + +Do not weaken authentication to work around an error. If a Server never +becomes healthy, the image is probably older than the release this stack +needs; report that instead of disabling the healthcheck. Only if I explicitly +ask for an unauthenticated cluster, use the "Running without authentication" +section instead. +``` + +</details> + +--- + +## Hubble for the 3-Node Cluster + +`docker-compose-hubble.yml` defines only the Hubble service. It joins the +cluster's external network (`hugegraph-net` by default, override with +`HUGEGRAPH_NETWORK`) and has no `depends_on` on cluster services, so +starting, stopping, or upgrading Hubble never recreates or restarts PD, +Store, or Server containers. Hubble reads the cluster topology from +`hugegraph-hubble-3x3.properties`; adjust that file when attaching to a +cluster with different hostnames. Its `pd.server` is a single PD address +with no failover, so if that PD is down the operations view goes blind even +though the cluster still has quorum — repoint it at a surviving PD. The +operations view also reports one `SERVER` node, not three: it describes the +replica Hubble is currently talking to, not the whole Server tier. The add-on +stores Hubble's H2 database and uploaded files in named volumes +`hugegraph-hubble-db` (mounted at `/hubble/db`) and +`hugegraph-hubble-upload-files`. Those names are explicit, so the attach +and combined Compose projects share the same physical volumes. Hubble's +default H2 URL would write `/hubble/db.mv.db` (outside that mount); the +add-on sets `SPRING_DATASOURCE_URL` so the database file lives inside +`/hubble/db`. Recreating the container does not discard that state. + +Sign in at `http://localhost:8088` as `admin` with the +`HUGEGRAPH_ADMIN_PASSWORD` from `docker/.env`. Hubble binds to host +loopback by default (`HUBBLE_PUBLISH_HOST`, same caveats as the +single-node setup). + +Which flow to use: pick the attach flow when you do not have the cluster's +`docker/.env` — the add-on carries no `:?` guards, so it is the only flow +that runs without those credentials, which is what you want against a +cluster someone else started. Otherwise use the combined flow, including to +add Hubble to an already-running cluster (`up -d --no-deps hubble`). + +The two flows below create Hubble in different Compose projects, so manage +Hubble with the same flags you started it with: the attach flow always uses +`-p hugegraph-hubble -f docker-compose-hubble.yml`, the combined flow always +uses both `-f` flags. The explicit `-p` keeps the attach project independent +of the directory name and of other Compose projects. + +> [!IMPORTANT] +> Do not drop the `-p` from the attach flow. Without it Compose names the +> project after the current directory (`docker`), so Hubble starts in a +> third project that none of the commands below manage: `down` reports +> nothing to remove while `hg-hubble` keeps running, and every later `up` +> in either flow fails on the container name. If that happens, find it with +> `docker ps --filter name=hg-hubble` and remove it with +> `docker rm -f hg-hubble`. + +Run one Hubble per host: the single-node stack and both add-on flows all +publish `127.0.0.1:8088` and name their container `hg-hubble`. The two +add-on flows are therefore mutually exclusive — starting one while the +other's Hubble exists fails with a container-name conflict, so remove the +Hubble of the flow you are leaving before switching. The two directions are +not symmetric: + +```bash +cd docker +# leaving the attach flow (removes only Hubble) +docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down + +# leaving the combined flow — remove ONLY Hubble; a plain `down` with both +# -f flags would stop all ten containers just to move one. +docker compose -f docker-compose-3pd-3store-3server.yml \ + -f docker-compose-hubble.yml rm -sf hubble +``` + +### Attach to a running cluster + +With the 3-node cluster already up. Hubble's two volumes are `external`, so +Compose refuses to start until they exist. Whoever ran the quickstart setup +block already created them; create them here too when attaching to a cluster +someone else started, which is the case this flow exists for. Both commands +are safe to repeat. + +```bash +cd docker + +# Match HUBBLE_DB_VOLUME and HUBBLE_UPLOAD_VOLUME if the cluster owner set +# them to something other than these defaults. +for vol in "${HUBBLE_DB_VOLUME:-hugegraph-hubble-db}" \ + "${HUBBLE_UPLOAD_VOLUME:-hugegraph-hubble-upload-files}"; do + docker volume inspect "${vol}" >/dev/null 2>&1 || + docker volume create "${vol}" >/dev/null +done + +docker compose -p hugegraph-hubble -f docker-compose-hubble.yml up -d +``` + +Lifecycle commands in this flow operate on Hubble alone and leave the +cluster and the external network in place: + +```bash +cd docker +docker compose -p hugegraph-hubble -f docker-compose-hubble.yml ps +docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down +``` + +To remove everything in this flow, take down Hubble first, then the cluster. +The cluster file's `:?` guards fire on `down` too, so this needs `docker/.env` +or the same two variables in the environment. If you attached to a cluster +someone else started and never had those credentials, remove only Hubble with +the command above and leave the cluster to whoever owns it. + +```bash +cd docker +docker compose -p hugegraph-hubble -f docker-compose-hubble.yml down +docker compose -f docker-compose-3pd-3store-3server.yml down -v + +# Hubble's volumes are external, so no `down` removes them. Drop them +# explicitly when you want the H2 database and uploaded files gone, or a +# later deployment reattaches the old state. +docker volume rm hugegraph-hubble-db hugegraph-hubble-upload-files Review Comment: 🧹 This teardown only removes the default volume names. Evidence: the add-on supports HUBBLE_DB_VOLUME and HUBBLE_UPLOAD_VOLUME, but this command hard-codes the defaults, so a customized deployment retains its H2 and upload data despite the instruction to drop them. Use the same variables with defaults or say custom volumes must be removed separately. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
