I do this way: - Launch a new instance by clicking on the slave instance and choose *launch more like this * *- *Once its launched, ssh into it and add the master public key to .ssh/authorized_keys - Add the slaves internal IP to the master's conf/slaves file - do sbin/start-all.sh and it will show up along with other slaves.
Thanks Best Regards On Thu, May 28, 2015 at 12:29 PM, nizang <ni...@windward.eu> wrote: > hi, > > I'm working on spark standalone system on ec2, and I'm having problems on > resizing the cluster (meaning - adding or removing slaves). > > In the basic ec2 scripts > (http://spark.apache.org/docs/latest/ec2-scripts.html), there's only > script > for lunching the cluster, not adding slaves to it. On the spark-standalone > page > ( > http://spark.apache.org/docs/latest/spark-standalone.html#cluster-launch-scripts > ), > I can see only options for stopping and starting slaves, not adding them. > > What I try to do now (as a bad workaround...), is the following: > > 1) Go to the ec2 UI, create image from the current slave > 2) Lunch new instance based on this image > 3) Copy the public DNS of this slave > 4) SSH to the master, and edit the file "/root/spark-ec2/ec2-variables.sh", > and add the DNS to the "export SLAVES" variables > 5) Running the script /root/spark-ec2/setup.sh > > After doing the above steps, I can see the new slave in the UI (8080 port) > of the master. However, this solution is bad for many reasons: > > 1) It requires many manual steps > 2) It requires stopping and starting the cluster > 3) There's no auto-detection in case slave stopped > > and many other reasons... > > oes anybody have another idea on how to add/remove slaves for standalone on > a simple and safe way? > > thanks, nizan > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Adding-slaves-on-spark-standalone-on-ec2-tp23064.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > >
