Re: spark k8s submit

> admission webhook "validation.gatekeeper.sh" denied the request: First, your error is not related to Spark; it's related to your cluster policies created by Gatekeeper . However, yes, you can fix it by providing a pod template to the Spark

Re: spark k8s submit

Could you file an official Spark JIRA issue with those reproducers (your CLI command, YAML files, error messages)? In addition, it would be helpful if you can describe how to set up your K8s clusters to make sure that it's not a K8s cluster issue. After having a JIRA, we can continue on that JIRA

Re: spark k8s submit

Thanks, Dongjoon for the details. almost similar yaml file I have added for the template reference file and I am getting below error Exception in thread "main" io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: https://k8sURL/api/v1/namespaces/namespace1/pods

Re: spark k8s submit

Could you elaborate what you mean by `not working`? > but it's not working. For the following question, Spark expects a normal Pod YAML file. You may want to take a look at the Apache Spark GitHub repository. > I do not have a sample template file For example, the following files are used duri

spark k8s submit

Hi, I am trying to run Spark on the Kubernetes cluster, but that cluster has certain validation to deploy any pod that is not allowing me to run my Spark submit. for example, I need to add liveness, readiness probes and certain security capability restrictions, which we usually do for all outer p