Re: Detect executor core count

tch { >> >> case _: NoSuchElementException => >> >> // If spark.executor.cores is not defined, get the cores per JVM >> >> import spark.implicits._ >> >> val numMachineCores = spark.range(0, 1) >> >> .map(_ =>

Re: Detect executor core count

ined, get the cores per JVM > > import spark.implicits._ > > val numMachineCores = spark.range(0, 1) > > .map(_ => > java.lang.Runtime.getRuntime.availableProcessors).collect.head > > numMachineCores > > } > > > > Thank you, Ilya > > > > *From:* Andrew

RE: Detect executor core count

spark.range(0, 1) .map(_ => java.lang.Runtime.getRuntime.availableProcessors).collect.head numMachineCores } Thank you, Ilya From: Andrew Melo Sent: Tuesday, June 18, 2019 11:32 AM To: dev Subject: Detect executor core count Hello, Is there a way to detect the number of cores allocated for an executor withi

Detect executor core count

Hello, Is there a way to detect the number of cores allocated for an executor within a java-based InputPartitionReader? Thanks! Andrew