Re: Error while running Spark on K8s

2021-01-04 Thread Prashant Sharma
Can you please paste the full exception trace, and mention spark and k8s version? On Mon, Jan 4, 2021 at 6:19 PM Sachit Murarka wrote: > Hi Prashant > > Thanks for the response! > > I created the service account with the permissions and following is the > command: > > spark-submit --deploy-mode

Re: Error while running Spark on K8s

2021-01-04 Thread Sachit Murarka
Hi Prashant Thanks for the response! I created the service account with the permissions and following is the command: spark-submit --deploy-mode cluster --master k8s://http://ip:port --name "sachit" --conf spark.kubernetes.pyspark.pythonVersion=3 --conf spark.kubernetes.namespace=spark-test -

Re: Error while running Spark on K8s

2021-01-04 Thread Prashant Sharma
Hi Sachit, Can you give more details on how did you run? i.e. spark submit command. My guess is, a service account with sufficient privilege is not provided. Please see: http://spark.apache.org/docs/latest/running-on-kubernetes.html#rbac Thanks, On Mon, Jan 4, 2021 at 5:27 PM Sachit Murarka wro