Re: spark EC2 bring-up problems

2014-06-12 Thread Toby Douglass
On Thu, Jun 12, 2014 at 9:10 PM, Zongheng Yang wrote: > Hi Toby, > > It is usually the case that even if the EC2 console says the nodes are > up, they are not really fully initialized. For 16 nodes I have found > `--wait 800` to be the norm that makes things work. > It seems so! resume worked f

Re: spark EC2 bring-up problems

2014-06-12 Thread Zongheng Yang
Hi Toby, It is usually the case that even if the EC2 console says the nodes are up, they are not really fully initialized. For 16 nodes I have found `--wait 800` to be the norm that makes things work. In my previous experience I have found this to be the culprit, so if you immediately do 'launch

Re: spark EC2 bring-up problems

2014-06-12 Thread Toby Douglass
On Thu, Jun 12, 2014 at 8:50 PM, Nicholas Chammas < nicholas.cham...@gmail.com> wrote: > Yes, you need Python 2.7 to run spark-ec2 and most AMIs come with 2.6 > Ah, yes - I mean to say, Amazon Linux. > .Have you tried either: > >1. Retrying launch with the --resume option? >2. Increasing

Re: spark EC2 bring-up problems

2014-06-12 Thread Nicholas Chammas
Yes, you need Python 2.7 to run spark-ec2 and most AMIs come with 2.6. Have you tried either: 1. Retrying launch with the --resume option? 2. Increasing the value of the --wait option? Nick ​ On Thu, Jun 12, 2014 at 3:38 PM, Toby Douglass wrote: > Gents, > > I have been bringing up a c