Re: [Spark in Kubernetes] Question about running in client mode

Hi Attila, Ah that makes sense. Thanks for the clarification! Best, Shiqi On Mon, Apr 26, 2021 at 8:09 PM Attila Zsolt Piros < piros.attila.zs...@gmail.com> wrote: > Hi Shiqi, > > In case of client mode the driver runs locally: in the same machine, even > in the same process, of the spark submi

Re: [Spark in Kubernetes] Question about running in client mode

Hi Shiqi, In case of client mode the driver runs locally: in the same machine, even in the same process, of the spark submit. So if the application was submitted in a running POD then the driver will be running in a POD and when outside of K8s then it will be running outside. This is why there is

[Spark in Kubernetes] Question about running in client mode

Hi Spark User group, I have a couple of quick questions about running Spark in Kubernetes between different deploy modes. As specified in https://spark.apache.org/docs/latest/running-on-kubernetes.html#client-mode, since Spark 2.4, client mode support is available when running in Kubernetes, and