ruanwenjun opened a new issue, #7495: URL: https://github.com/apache/kyuubi/issues/7495
### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) ### Search before asking - [x] I have searched in the [issues](https://github.com/apache/kyuubi/issues?q=is%3Aissue) and found no similar issues. ### Describe the bug When Kyuubi submits Spark applications to Kubernetes, `KubernetesUtils` generates the Spark driver pod name and executor pod name prefix from the Spark application name and engine reference id. The current length check only considers the Kubernetes pod name limit. For example, the driver pod name is allowed up to 253 characters, and the executor pod name prefix reserves only Spark's executor suffix. However, kubelet creates the pod log directory with the following format: ```text namespace_podName_podUID ``` See Kubernetes' `BuildPodLogsDirectory` implementation: https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/kuberuntime/helpers.go#L223 As a result, a pod name that is valid for Kubernetes can still make the kubelet log directory name too long after the namespace, separators, and pod UID are appended. In that case, Spark submission can succeed far enough to create the pod, but the kubelet fails when creating the pod log directory on the node because the directory name exceeds the filesystem name length limit. Kyuubi should take the kubelet pod log directory format into account when generating Spark driver pod names and executor pod name prefixes for Kubernetes. ### Affects Version(s) master ### Kyuubi Server Log Output ```logtalk N/A ``` ### Kyuubi Engine Log Output ```logtalk N/A ``` ### Kyuubi Server Configurations ```yaml spark.master: k8s://... spark.submit.deployMode: cluster # The issue is easier to reproduce with a long spark.app.name and/or a long Kubernetes namespace. ``` ### Kyuubi Engine Configurations ```yaml N/A ``` ### Additional context The fix should preserve user-specified `spark.kubernetes.driver.pod.name` and `spark.kubernetes.executor.podNamePrefix`, but generated names should leave enough room for the namespace, the two `_` separators, the pod UID, and Spark's executor suffix where applicable. ### Are you willing to submit PR? - [x] Yes. I would be willing to submit a PR with guidance from the Kyuubi community to fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
