wangzhigang1999 opened a new issue, #7488: URL: https://github.com/apache/kyuubi/issues/7488
### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [X] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug When a **RUNNING** batch is canceled via `DELETE /api/v1/batches/{batchId}` (or the cancel button in the Web UI), the batch operation state correctly transitions to `CANCELED` and the Spark application is actually killed — yet the batch keeps reporting `appState = RUNNING` forever. The REST API permanently returns `state=CANCELED` together with `appState=RUNNING`, which is contradictory and misleading (e.g. the Web UI shows a canceled batch whose application still looks alive). The application really is gone: the driver Pod and driver Service are deleted, and the `engine-ui` reverse proxy to the batch's `appUrl` returns HTTP 502. So this is purely stale metadata, not an orphaned/leaked application. **Steps to reproduce** 1. Submit a long-running Spark batch in cluster mode and wait until it reaches `RUNNING` (so `appState=RUNNING` and `appUrl` are populated). 2. `DELETE /api/v1/batches/{batchId}` — the response is `CloseBatchResponse(success=true)`. 3. `GET /api/v1/batches/{batchId}`. **Actual result** — `appState` stays `RUNNING` indefinitely, even though the driver Pod/Service are already gone: ```json { "id": "723d0f90-8a0d-49eb-9b5d-8a7a84e35787", "name": "claude-sparkpi-resident", "state": "CANCELED", "appState": "RUNNING", "appId": "spark-d1a76dfdc1cd4836aaabebc8b09d616d", "appUrl": "http://...-driver-svc.<ns>.svc:4040", "endTime": 1780411775395 } ``` **Expected result** — once a batch is canceled, its application has been killed, so `appState` should report a terminated state (e.g. `KILLED`) rather than a stale `RUNNING`. ### Affects Version(s) master (verified on commit `7067644e1`) and 1.10.1. The affected code path exists in all released versions; no branch currently fixes it. ### Kyuubi Server Log Output _No response_ ### Kyuubi Engine Log Output _No response_ ### Kyuubi Server Configurations _No response_ ### Kyuubi Engine Configurations _No response_ ### Additional context **Root cause** In `BatchJobSubmission.close()`, the kill-success branch sets the operation state to `CANCELED` and persists metadata via `updateBatchMetadata()` without re-polling the application info. `updateBatchMetadata()` writes `engineState = getAppState(state, appInfo.state)`, where `appInfo.state` is the last-polled value (`RUNNING`). `getAppState(...)` only coerces a non-terminated application state to `UNKNOWN` when `opState == ERROR`; it leaves `CANCELED` untouched and returns `RUNNING` as-is. Because status polling stops once the operation is closed, the stale `RUNNING` is persisted and reported forever. The same gap applies to the peer-instance-closed cancellation path (`peerInstanceClosed`), which also reaches `CANCELED` carrying the last-polled application state. The kill-*failed* branch is unaffected, since it re-polls and maps `KILLED`/`NOT_FOUND` appropriately. The earlier related fixes #6722 and #6668 do not cover this cancel path. **Proposed fix** When the operation state is `CANCELED` and the application state is not yet terminated, reconcile it to `KILLED` (the application has been killed), mirroring the existing `ERROR -> UNKNOWN` handling in `getAppState`. ### Are you willing to submit PR? - [X] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
