On Mon, Oct 7, 2019 at 20:49 ayan guha wrote:
> HI
>
> I think you are mixing terminologies here. Loosely speaking, Master
> manages worker machines. Each worker machine can run one or more processes.
> A process can be a driver or executor. You submit applications to the
> master. Each applicati
HI
I think you are mixing terminologies here. Loosely speaking, Master manages
worker machines. Each worker machine can run one or more processes. A
process can be a driver or executor. You submit applications to the master.
Each application will have driver and executors. Master will decide where
Hi
On Mon, Oct 7, 2019 at 19:20 Amit Sharma wrote:
> Thanks Andrew but I am asking specific to driver memory not about
> executors memory. We have just one master and if each jobs driver.memory=4g
> and master nodes total memory is 16gb then we can not execute more than 4
> jobs at a time.
I u
Thanks Andrew but I am asking specific to driver memory not about executors
memory. We have just one master and if each jobs driver.memory=4g and
master nodes total memory is 16gb then we can not execute more than 4 jobs
at a time.
On Monday, October 7, 2019, Andrew Melo wrote:
> Hi Amit
>
> On
Hi Amit
On Mon, Oct 7, 2019 at 18:33 Amit Sharma wrote:
> Can you please help me understand this. I believe driver programs runs on
> master node
If we are running 4 spark job and driver memory config is 4g then total 16
> 6b would be used of master node.
This depends on what master/deploy mo
