Hi Zoltan,

If running on YARN, the YARN NodeManager starts executors.  I don't think
there's a 100% precise way for the Spark executor way to know how many
resources are allotted to it.  It can come close by looking at the Spark
configuration options used to request it (spark.executor.memory and
spark.yarn.executor.memoryOverhead), but it can't necessarily for the
amount that YARN has rounded up if those configuration properties
(yarn.scheduler.minimum-allocation-mb and
yarn.scheduler.increment-allocation-mb) are not present on the node.

-Sandy

-Sandy

On Mon, Mar 23, 2015 at 5:08 PM, Zoltán Zvara <zoltan.zv...@gmail.com>
wrote:

> Let's say I'm an Executor instance in a Spark system. Who started me and
> where, when I run on a worker node supervised by (a) Mesos, (b) YARN? I
> suppose I'm the only one Executor on a worker node for a given framework
> scheduler (driver). If I'm an Executor instance, who is the closest object
> to me who can tell me how many resources do I have on (a) Mesos, (b) YARN?
>
> Thank you for your kind input!
>
> Zvara Zoltán
>
>
>
> mail, hangout, skype: zoltan.zv...@gmail.com
>
> mobile, viber: +36203129543
>
> bank: 10918001-00000021-50480008
>
> address: Hungary, 2475 Kápolnásnyék, Kossuth 6/a
>
> elte: HSKSJZ (ZVZOAAI.ELTE)
>

Reply via email to