Re: Spark ec2 cluster lost worker

2015-06-24 Thread Kelly, Jonathan
Jonathan Kelly From: Anny Chen mailto:anny9...@gmail.com>> Date: Wednesday, June 24, 2015 at 6:15 PM To: Jonathan Kelly mailto:jonat...@amazon.com>> Cc: "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: Spark ec2 c

Re: Spark ec2 cluster lost worker

2015-06-24 Thread Anny Chen
Hi Jonathan, Thanks for this information! I will take a look into it. However is there a way to reconnect the lost node? Or there's no way that I could do to find back the lost worker? Thanks! Anny On Wed, Jun 24, 2015 at 6:06 PM, Kelly, Jonathan wrote: > Just curious, would you be able to use

Re: Spark ec2 cluster lost worker

2015-06-24 Thread Kelly, Jonathan
Just curious, would you be able to use Spark on EMR rather than on EC2? Spark on EMR will handle lost nodes for you, and it will let you scale your cluster up and down or clone a cluster (its config, that is, not the data stored in HDFS), among other things. We also recently announced official supp