re than 1 workers per machine
>
>
> Sent from Samsung Mobile
>
>
> Original message
> From: Sandy Ryza
> Date:2015/06/10 21:31 (GMT+00:00)
> To: Evo Eftimov
> Cc: maxdml ,user@spark.apache.org
> Subject: Re: Determining number of executors within
Sent from Samsung Mobile
Original message
From: maxdml
Date:2015/06/10 19:56 (GMT+00:00)
To: user@spark.apache.org
Subject: Re: Determining number of executors within RDD
Actually this is somehow confusing for two reasons:
- First, the option 'spark.executor.instances',
>
>
> Sent from Samsung Mobile
>
>
> Original message
> From: maxdml
> Date:2015/06/10 19:56 (GMT+00:00)
> To: user@spark.apache.org
> Subject: Re: Determining number of executors within RDD
>
> Actually this is somehow confusing for two reasons
/executot
Sent from Samsung Mobile
Original message From: maxdml
Date:2015/06/10 19:56 (GMT+00:00)
To: user@spark.apache.org Subject: Re: Determining number
of executors within RDD
Actually this is somehow confusing for two reasons:
- First, the option
on-options-td13692.html)
Could anyone clarify this? :-)
Thanks.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Determining-number-of-executors-within-RDD-tp15554p23262.html
Sent from the Apache Spark User List mailing list archive at
Note that this property is only available for YARN
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Determining-number-of-executors-within-RDD-tp15554p23256.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
partitions.
And of course if you want you can determine the number of executor as well
by setting 'spark.executor.instances' property in 'sparkConf' object.
Thank you.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Determining-number-of
spark-user-list.1001560.n3.nabble.com/Determining-number-of-executors-within-RDD-tp15554p23234.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
Hi,
I want implement an RDD wherein the decision of number of partitions is
based on the number of executors that have been set up. Is there some way I
can determine the number of executors within the getPartitions() call?
