wangzhigang1999 commented on code in PR #7489:
URL: https://github.com/apache/kyuubi/pull/7489#discussion_r3346329252
##########
kyuubi-server/src/main/scala/org/apache/kyuubi/operation/BatchJobSubmission.scala:
##########
@@ -195,6 +195,13 @@ class BatchJobSubmission(
withOperationLog(error(s"Batch $batchId state is $opState," +
s" but the application state is $appState and not terminated, set to
UNKNOWN."))
ApplicationState.UNKNOWN
+ } else if (opState == OperationState.CANCELED &&
!ApplicationState.isTerminated(appState)) {
+ // A canceled batch has had its application killed, so the last-polled
application state
+ // (e.g. RUNNING) is stale. Without reconciling it, this stale state
would be persisted and
+ // reported by the REST API forever, since polling stops once the
operation is closed.
+ withOperationLog(info(s"Batch $batchId state is $opState," +
+ s" but the application state is $appState and not terminated, set to
KILLED."))
+ ApplicationState.KILLED
Review Comment:
Good point. Querying the RM helps, but it does not fully fix the stale
metadata issue by itself: application kill is asynchronous, so an immediate RM
query may still report RUNNING/PENDING. Since Kyuubi stops polling once the
batch operation is closed, persisting that non-terminal state would make the
REST API report it forever.
I think UNKNOWN is not a good fallback here either, because UNKNOWN is not a
terminal ApplicationState in Kyuubi, so it would still leave clients with a
non-terminal state that will never converge.
So the intended logic is: after killApplication succeeds, prefer the RM
state if it has already converged to a terminal state; otherwise, for a
CANCELED operation whose last observed app state is still non-terminal,
reconcile it to KILLED instead of persisting stale RUNNING/PENDING. The KILLED
fallback is only used after the kill-success path, not for arbitrary canceled
metadata. I can re-poll once in the kill-success branch before applying the
fallback and clarify the comment.
WDYT？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
