> eabour
>
>
> *From:* eab...@163.com
> *Date:* 2023-10-19 14:28
> *To:* Nagatomi Yasukazu ; user @spark
>
> *Subject:* Re: Re: Running Spark Connect Server in Cluster Mode on
> Kubernetes
> Hi all,
>
> Has the spark connect server running on k8s functiona
-10-19 14:28
To: Nagatomi Yasukazu; user @spark
Subject: Re: Re: Running Spark Connect Server in Cluster Mode on Kubernetes
Hi all,
Has the spark connect server running on k8s functionality been implemented?
From: Nagatomi Yasukazu
Date: 2023-09-05 17:51
To: user
Subject: Re: Running Spark
Hi all,
Has the spark connect server running on k8s functionality been implemented?
From: Nagatomi Yasukazu
Date: 2023-09-05 17:51
To: user
Subject: Re: Running Spark Connect Server in Cluster Mode on Kubernetes
Dear Spark Community,
I've been exploring the capabilities of the
Dear Spark Community,
I've been exploring the capabilities of the Spark Connect Server and
encountered an issue when trying to launch it in a cluster deploy mode with
Kubernetes as the master.
While initiating the `start-connect-server.sh` script with the `--conf`
parameter for `spark.master` and
Hello Mich,
Thank you for your questions. Here are my responses:
> 1. What investigation have you done to show that it is running in local
mode?
I have verified through the History Server's Environment tab that:
- "spark.master" is set to local[*]
- "spark.app.id" begins with local-xxx
- "spark.s
personally I have not used this feature myself. However, some points
1. What investigation have you done to show that it is running in local
mode?
2. who has configured this kubernetes cluster? Is it supplied by a cloud
vendor?
3. Confirm that you have configured Spark Connect Serv
Hi Cley,
Thank you for taking the time to respond to my query. Your insights on
Spark cluster deployment are much appreciated.
However, I'd like to clarify that my specific challenge is related to
running the Spark Connect Server on Kubernetes in Cluster Mode. While I
understand the general deplo
Hi Nagatomi,
Use Apache imagers, then run your master node, then start your many
slavers. You can add a command line in the docker files to call for the
master using the docker container names in your service composition if you
wish to run 2 masters active and standby follow the instructions in the
