Hi Shiqi, In case of client mode the driver runs locally: in the same machine, even in the same process, of the spark submit.
So if the application was submitted in a running POD then the driver will be running in a POD and when outside of K8s then it will be running outside. This is why there is no config mentioned for this. >From the deploy mode in general you can read here: https://spark.apache.org/docs/latest/submitting-applications.html Best Regards, Attila On Tue, Apr 27, 2021 at 12:03 AM Shiqi Sun <jack.sun...@gmail.com> wrote: > Hi Spark User group, > > I have a couple of quick questions about running Spark in Kubernetes > between different deploy modes. > > As specified in > https://spark.apache.org/docs/latest/running-on-kubernetes.html#client-mode, > since Spark 2.4, client mode support is available when running in > Kubernetes, and it says "when your application runs in client mode, the > driver can run inside a pod or on a physical host". Then here come the > questions. > > 1. If I understand correctly, in cluster mode, the driver is also running > inside a k8s pod. Then, what's the difference between running it in cluster > mode, versus running it in client mode when I choose to run my driver in a > pod? > > 2. What does it mean by "running driver on a physical host"? Does it mean > that it runs outside of the k8s cluster? What config should I pass to spark > submit so that it runs this way, instead of running my driver into a k8s > pod? > > Thanks! > > Best, > Shiqi >
