Thanks a lot Andrew! Yeah I actually realized that later. I made a silly
mistake here.
On Mon, Oct 20, 2014 at 6:03 PM, Andrew Ash wrote:
> Hi Anny, SPARK_WORKER_INSTANCES is the number of copies of spark workers
> running on a single box. If you change the number you change how the
> hardware
Hi Anny, SPARK_WORKER_INSTANCES is the number of copies of spark workers
running on a single box. If you change the number you change how the
hardware you have is split up (useful for breaking large servers into <32GB
heaps each which perform better) but doesn't change the amount of hardware
you h
Hi,
I have a question about the worker_instances setting and worker_cores
setting in aws ec2 cluster. I understand it is a cluster and the default
setting in the cluster is
*SPARK_WORKER_CORES = 8
SPARK_WORKER_INSTANCES = 1*
However after I changed it to
*SPARK_WORKER_CORES = 8
SPARK_WORKER_INS
