Thank you guys for the detailed answer.
Akhil, yes I would like to have a try of your tool. Is it open-sourced?
2014-05-17 17:55 GMT+02:00 Mayur Rustagi :
> A better way would be use Mesos (and quite possibly Yarn in 1.0.0).
> That will allow you to add nodes on the fly & leverage it for Spark.
A better way would be use Mesos (and quite possibly Yarn in 1.0.0).
That will allow you to add nodes on the fly & leverage it for Spark.
Frankly Standalone mode is not meant to handle those issues. That said we
use our deployment tool as stopping the cluster for adding nodes is not
really an issue
Thanks for the info about adding/removing nodes dynamically. That's
valuable.
2014년 5월 16일 금요일, Akhil Das님이 작성한 메시지:
> Hi Han :)
>
> 1. Is there a way to automatically re-spawn spark workers? We've
> situations where executor OOM causes worker process to be DEAD and it does
> not came back automa
