I'm following the tutorial about Apache Spark on EC2. The output is the
following:
$ ./spark-ec2 -i ../spark.pem -k spark --copy launch spark-training
Setting up security groups...
Searching for existing cluster spark-training...
Latest Spark AMI: ami-19474270
Launching instances...
Launched 5 slaves in us-east-1d, regid = r-59a0d4b6
Launched master in us-east-1d, regid = r-9ba2d674
Waiting for instances to start up...
Waiting 120 more seconds...
Copying SSH key ../spark.pem to master...
ssh: connect to host ec2-54-152-15-165.compute-1.amazonaws.com port 22:
Connection refused
Error connecting to host Command 'ssh -t -o StrictHostKeyChecking=no -i
../spark.pem [email protected] 'mkdir -p ~/.ssh''
returned non-zero exit status 255, sleeping 30
ssh: connect to host ec2-54-152-15-165.compute-1.amazonaws.com port 22:
Connection refused
Error connecting to host Command 'ssh -t -o StrictHostKeyChecking=no -i
../spark.pem [email protected] 'mkdir -p ~/.ssh''
returned non-zero exit status 255, sleeping 30
ssh: Could not resolve hostname ec2-54-152-15-165.compute-1.amazonaws.com:
Name or service not known
Error connecting to host Command 'ssh -t -o StrictHostKeyChecking=no -i
../spark.pem [email protected] 'mkdir -p ~/.ssh''
returned non-zero exit status 255, sleeping 30
ssh: connect to host ec2-54-152-15-165.compute-1.amazonaws.com port 22:
Connection refused
Traceback (most recent call last):
File "./spark_ec2.py", line 925, in <module>
main()
File "./spark_ec2.py", line 766, in main
setup_cluster(conn, master_nodes, slave_nodes, zoo_nodes, opts, True)
File "./spark_ec2.py", line 406, in setup_cluster
ssh(master, opts, 'mkdir -p ~/.ssh')
File "./spark_ec2.py", line 712, in ssh
raise e
subprocess.CalledProcessError: Command 'ssh -t -o StrictHostKeyChecking=no
-i ../spark.pem [email protected] 'mkdir -p
~/.ssh'' returned non-zero exit status 255
However, I can see the six instances created on my EC2 console, and I could
even get the name of the master. I'm not sure how to fix the ssh issue (my
region is US EST).
