Kousuke Saruta created SPARK-57895:
--------------------------------------
Summary: Implement initial credential delivery for newly
registered executors
Key: SPARK-57895
URL: https://issues.apache.org/jira/browse/SPARK-57895
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 4.3.0
Reporter: Kousuke Saruta
This sub-task is part of the umbrella
[SPARK-57703|https://issues.apache.org/jira/browse/SPARK-57703] (OIDC
Credential Propagation).
h2. Problem
Executors that register after the initial credential acquisition (e.g., dynamic
allocation) have no credentials until the next RPC broadcast or task dispatch.
h2. Goal
Ensure newly-registered executors receive current credentials immediately on
registration.
h2. Scope
* Add {{userCredentials: Option[Array[Byte]]}} field to {{SparkAppConfig}}
* On executor registration, include the latest credentials from the
{{AtomicReference}} in the response
* Executor processes received credentials on startup, populating the credential
store
h2. Acceptance criteria
* A late-registering executor receives current credentials without waiting for
a refresh cycle or task dispatch.
* Mirrors existing behavior for {{hadoopDelegationCreds}}.
* Unit test: executor registers after credentials are acquired; verify
credentials are present in credential store immediately.
Reference: [SPIP design document Appendix
B|https://docs.google.com/document/d/1usJKncCPMiyFUg7aIdpZ0HQsklXIHow_sU_6dfFMjN0/edit].
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
