Research Staff Member
IBM T. J. Watson Research Center
http://researcher.ibm.com/person/us-wtan
From: innowireless TaeYun Kim
To: ,
Date: 07/16/2014 05:04 AM
Subject:RE: executor-cores vs. num-executors
Thanks.
Really, now I compare a stage data of the two jobs, ‘core7-exec3
Thanks.
Really, now I compare a stage data of the two jobs, ‘core7-exec3’ spends about
12.5 minutes more than ‘core2-exec12’ on GC.
From: Nishkam Ravi [mailto:nr...@cloudera.com]
Sent: Wednesday, July 16, 2014 5:28 PM
To: user@spark.apache.org
Subject: Re: executor-cores vs. num
I think two small JVMs would often beat a large one due to lower GC
overhead.
Hi,
On running yarn-client mode, the following options can be specified:
l --executor-cores
l --num-executors
If we have following machines:
l 3 data nodes
l 8 cores each node
Which is the better?
1. --executor-cores 7 --num-executors 3 (more core for each executor,
leavi
