Hi,
On Sat, Jan 17, 2015 at 3:05 AM, Shuai Zheng <[email protected]> wrote:
>
> Can you share more information about how do you do that? I also have
> similar question about this.
>
Not very proud about it ;-), but here you go:
// find the number of workers available to us.
val _runCmd = scala.util.Properties.propOrElse("sun.java.command", "")
val numCoresRe = ".*--executor-cores ([0-9]+) --num-executors ([0-9]+).*".r
val totalNumCores = _runCmd match {
case numCoresRe(coresPerExecutor, numExecutors) =>
coresPerExecutor.toInt * numExecutors.toInt
case _ =>
0
}
if (totalNumCores > 0)
logger.debug("total number of cores: " + totalNumCores)
else
logger.warn("could not extract number of cores from run command: " + _runCmd)
Tobias
