On Sun, Feb 8, 2015 at 10:26 PM, java8964 wrote:
> standalone one box environment, if I want to use all 48G memory allocated to
> worker for my application, I should ask 48G memory for the executor in the
> spark shell, right? Because 48G is too big for a JVM heap in normal case, I
> can and shoul
t one partition (or one task for it) will be run
by one executor? Is that one executor will run the task sequentially, but job
concurrency comes from that multi executors could run synchronous, right?
Thanks
> From: so...@cloudera.com
> Date: Sun, 8 Feb 2015 20:53:15 +
> Subjec
I think I have this right:
You will run one executor per application per worker. Generally there
is one worker per machine, and it manages all of the machine's
resources. So if you want one app to use this whole machine you need
to ask for 48G and 24 cores. That's better than splitting up the
reso
Hi, I have some questions about how the spark run the job concurrently.
For example, if I setup the Spark on one standalone test box, which has 24 core
and 64G memory. I setup the Worker memory to 48G, and Executor memory to 4G,
and using spark-shell to run some jobs. Here is something confusing
