yandrey321 opened a new pull request, #11224:
URL: https://github.com/apache/ozone/pull/11224
## What changes were proposed in this pull request?
Adds two unauthenticated HTTP health endpoints to the S3 Gateway, served on
the
web admin server (default port `19878`), separate from the S3 data listener
(`9878`):
- `GET /health/live` → `200 OK` while the gateway process is up and its web
admin server can serve requests. It does **not** check OM reachability.
- `GET /health/ready` → `200 READY` only when the gateway can reach OM, and
`503 NOT READY` during startup or while OM is unreachable.
Readiness is backed by `S3GatewayReadinessProbe`, a single background thread
that periodically probes OM (`getServiceInfo`) and stores the result in a
volatile flag. The servlet only reads that flag, so `/health/ready` always
responds immediately — no OM RPC on the request path — and a load balancer
polling it can never be blocked by a slow or unreachable OM. The probe runs
its
OM call on a separate single-thread executor with an enforced deadline, so a
hung call flips the gateway to "not ready" rather than wedging the
scheduler. It
owns a dedicated `OzoneClient` (created via `OzoneClientCache.createClient`)
so
it exercises the same OM transport the gateway serves with, with S3 auth
disabled and a bounded OM RPC timeout.
The endpoints are registered with `addInternalServlet`, so no authentication
filter is mapped to them and they remain reachable by a load balancer without
Kerberos/SPNEGO, including in secure mode. Because they live on the admin
port,
they cannot collide with S3 bucket names or require SigV4 signing.
Wiring updated to use the new endpoints:
- Kubernetes S3G statefulsets: `livenessProbe` now points at `/health/live`
(was `/`) and a `readinessProbe` on `/health/ready` is added.
- HAProxy examples (`compose/common` and `compose/ozonesecure-ha`)
health-check
the readiness endpoint (`GET /health/ready`, expect `200`) with
`inter 2s rise 1` so backends are routed to promptly once ready.
- `compose/testlib.sh` gains `wait_for_s3g_ready()`, invoked from
`start_docker_env`, so acceptance tests don't start issuing S3 requests
while
HAProxy still has all backends `DOWN` (which would return `503`). It only
waits when the multi-instance HAProxy setup (`s3g1/s3g2/s3g3`) is present.
### Configuration
New `ozone.s3g.health-check.*` config group (`S3GatewayHealthCheckConfig`):
| Key | Default | Description |
| --- | --- | --- |
| `ozone.s3g.health-check.enabled` | `true` | Expose the `/health/live` and
`/health/ready` endpoints. |
| `ozone.s3g.health-check.probe.interval` | `10s` | How often the readiness
endpoint refreshes its cached OM reachability. |
| `ozone.s3g.health-check.probe.timeout` | `10s` | Upper bound on a single
background readiness probe. |
### Why are the changes needed?
When S3 Gateway is horizontally scaled behind a load balancer, the balancer
needs an HTTP health check to route S3 traffic only to gateways that can
actually serve it. The existing admin server had no dedicated
liveness/readiness
endpoints; the k8s liveness probe hit `/` and there was no readiness signal
at
all, so traffic could be sent to a gateway that is up but cannot reach OM.
Generated-by: Claude Code (Claude Opus 4.8)
## What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-16360
## How was this patch tested?
- New unit tests:
- `TestS3GatewayLivenessServlet` — liveness returns `200 OK`.
- `TestS3GatewayHealthCheck` — readiness reflects the probe flag (`200
READY`
vs `503 NOT READY`) and endpoints are gated by `health-check.enabled`.
- New acceptance test: `smoketest/s3/health.robot` exercises both endpoints;
`compose/ozone/test.sh` runs it.
- Verified the endpoints are served without authentication in secure mode
(registered via `addInternalServlet`).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
