Thanks Abhisehk I was able to resolve the issue.
I was building an assembly jar which has some unwanted spring and netty
classes. Because of which I was getting that exception.
Regards
Manish Gupta
--
Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/
overlay network.
Thanks and Regards,
Abhishek
From: manish gupta
Sent: 01 October 2019 PM 09:20
To: Prudhvi Chennuru (CONT)
Cc: user
Subject: Re: [External Sender] Spark Executor pod not getting created on
kubernetes cluster
Kube-api server logs are not enabled. I will enable and check and get back
Kube-api server logs are not enabled. I will enable and check and get back
on this.
Regards
Manish Gupta
On Tue, Oct 1, 2019 at 9:05 PM Prudhvi Chennuru (CONT) <
prudhvi.chenn...@capitalone.com> wrote:
> If you are passing the service account for executors as spark property
> then executor will
If you are passing the service account for executors as spark property then
executor will use the one you are passing not the default service account.
Did you check the api server logs?
On Tue, Oct 1, 2019 at 11:07 AM manish gupta
wrote:
> While launching the driver pod I am passing the service
While launching the driver pod I am passing the service account which has
cluster role and has all the required permissions to create a new pod. So
will driver pass the same details to API server while creating executor pod
OR executors will be created with default service account?
Regards
Manish
By default, executors use default service account in the namespace you are
creating the driver and executors so i am guessing that executors don't
have access to run on the cluster, if you check the kube-apisever logs you
will know the issue
and try giving privileged access to default service accou
Hi Prudhvi
I can see this issue consistently. I am doing a POC wherein I am trying to
create a dynamic spark cluster to run my job using spark submit on
Kubernetes. On Minikube it works fine but on rbac enabled kubernetes it
fails to launch executor pod. It is able to launch driver pod but not sur
Hi Manish,
Are you seeing this issue consistently or sporadically? and
when you say executors are not launched not even a single executor created
for that driver pod?
On Tue, Oct 1, 2019 at 1:43 AM manish gupta
wrote:
> Hi Team
>
> I am trying to create a spark cluster on kubernetes
Hi Team
I am trying to create a spark cluster on kubernetes with rbac enabled using
spark submit job. I am using spark-2.4.1 version.
Spark submit is able to launch the driver pod by contacting Kubernetes API
server but executor Pod is not getting launched. I can see the below
warning message in t
