[
https://issues.apache.org/jira/browse/SPARK-59518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18115420#comment-18115420
]
Anshul Baliga commented on SPARK-59518:
---------------------------------------
I'd like to work on this. The cause is in
ExecutorKubernetesCredentialsFeatureStep and the executor service account
config is optional so an explicitly empty value becomes Some("") and
.orElse(driverServiceAccount) never fires. I plan to filter out empty values
before the fallback and add a test to
ExecutorKubernetesCredentialsFeatureStepSuite. I'll keep the change independent
of #58343 (SPARK-58910) which touches the same file so either can land first.
Please let me know if you already have a fix in progress. Thank you!
> An empty spark.kubernetes.authenticate.executor.serviceAccountName is written
> to the executor pod instead of falling back to the driver account
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-59518
> URL: https://issues.apache.org/jira/browse/SPARK-59518
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 5.0.0
> Reporter: Yang Jie
> Priority: Major
>
> {{ExecutorKubernetesCredentialsFeatureStep}} treats
> {{spark.kubernetes.authenticate.executor.serviceAccountName=""}} as set: an
> empty value shadows a non-empty
> {{spark.kubernetes.authenticate.driver.serviceAccountName}} and is written
> into both {{serviceAccount}} and {{serviceAccountName}} on the executor pod,
> leaving it on the namespace default.
> The configuration's own doc says the driver account is the fallback "if this
> parameter is not setup", and the template side of SPARK-58910 counts an empty
> value as unset, matching Kubernetes' {{SetDefaults_PodSpec}}. So the same
> notion of "unset" is read two ways in one step.
> Aligning the write path means {{.filter(_.nonEmpty)}} on both {{lazy val}}s
> in that step. It is a small user-visible change: an empty executor value
> would fall back to the driver account instead of writing an empty one, so it
> is filed separately from SPARK-58910 rather than folded into a fix headed for
> the maintenance branches.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
