Thanks Tsuyoshi and Shixiong for the info. Awesome with more documentation
about the feature!
Was afraid that the node manager needed reconfiguration (and restart). Any
idea of how much resources will the shuffle service take on the node
manager? In a multi-tenant Hadoop cluster environment, it wo
I encountered the following issue when enabling dynamicAllocation. You may
want to take a look at it.
https://issues.apache.org/jira/browse/SPARK-4951
Best Regards,
Shixiong Zhu
2014-12-28 2:07 GMT+08:00 Tsuyoshi OZAWA :
> Hi Anders,
>
> I faced the same issue as you mentioned. Yes, you need to
Hi Anders,
I faced the same issue as you mentioned. Yes, you need to install
spark shuffle plugin for YARN. Please check following PRs which add
doc to enable dynamicAllocation:
https://github.com/apache/spark/pull/3731
https://github.com/apache/spark/pull/3757
I could run Spark on YARN with dyn
