yuqi1129 opened a new issue, #13066: URL: https://github.com/apache/gravitino/issues/13066
### Version main branch ### Describe what's wrong After an `OutOfMemoryError`, already-loaded REST endpoints and entity-store lookups can still succeed while other server operations fail. The main server's health endpoints nevertheless keep reporting HTTP 200 / UP: liveness only checks that an HTTP thread can answer, and readiness only probes the entity store. This allows load balancers and orchestration probes to continue treating a degraded server as healthy. A successful warm request does not establish that the process has recovered from OOM. ### Error message and/or stacktrace The reported symptom follows a server OOM; the original stack trace is not available. Relevant failure types include `java.lang.OutOfMemoryError: Metaspace` and `java.lang.OutOfMemoryError: Java heap space`. ### How to reproduce 1. Start the main server and warm `/api/health`, `/api/health/live`, and `/api/health/ready` while the entity store is reachable. 2. Cause a separate REST operation to throw `OutOfMemoryError`, directly or wrapped in another exception. A test resource can simulate this without exhausting the JVM. 3. Verify that a warm endpoint can still return 200. 4. Request the health endpoints again. They continue to report UP, including through `/health`, `/health/live`, `/health/ready`, and `/health.html`. ### Additional context Once the server observes OOM, the main health endpoints should report HTTP 503 with an explicit JVM failure until process restart. Ordinary application errors and temporary entity-store failures should not latch this state. Detection must account for wrapped errors and health-probe tasks whose failures are captured by futures, including tasks that fail after the probe times out. Errors swallowed entirely inside unrelated background tasks require separate detection; no JVM-wide OOM interception is assumed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
