you can comment out this function and Create a new one which will return
your ami-id and the rest of the script will run fine.
def get_spark_ami(opts):
instance_types = {
"m1.small": "pvm",
"m1.medium": "pvm",
"m1.large": "pvm",
"m1.xlarge": "pvm",
"t1.micro": "pvm",
"c1.medium": "pvm",
"c1.xlarge": "pvm",
"m2.xlarge": "pvm",
"m2.2xlarge": "pvm",
"m2.4xlarge": "pvm",
"cc1.4xlarge": "hvm",
"cc2.8xlarge": "hvm",
"cg1.4xlarge": "hvm",
"hs1.8xlarge": "hvm",
"hi1.4xlarge": "hvm",
"m3.xlarge": "hvm",
"m3.2xlarge": "hvm",
"cr1.8xlarge": "hvm",
"i2.xlarge": "hvm",
"i2.2xlarge": "hvm",
"i2.4xlarge": "hvm",
"i2.8xlarge": "hvm",
"c3.large": "pvm",
"c3.xlarge": "pvm",
"c3.2xlarge": "pvm",
"c3.4xlarge": "pvm",
"c3.8xlarge": "pvm"
}
if opts.instance_type in instance_types:
instance_type = instance_types[opts.instance_type]
else:
instance_type = "pvm"
print >> stderr,\
"Don't recognize %s, assuming type is pvm" % opts.instance_type
ami_path = "%s/%s/%s" % (AMI_PREFIX, opts.region, instance_type)
try:
ami = urllib2.urlopen(ami_path).read().strip()
print "Spark AMI: " + ami
except:
print >> stderr, "Could not resolve AMI at: " + ami_path
sys.exit(1)
return ami
Thanks
Best Regards
On Fri, Jun 6, 2014 at 2:14 AM, Matt Work Coarr <mattcoarr.w...@gmail.com>
wrote:
> How would I go about creating a new AMI image that I can use with the
> spark ec2 commands? I can't seem to find any documentation. I'm looking
> for a list of steps that I'd need to perform to make an Amazon Linux image
> ready to be used by the spark ec2 tools.
>
> I've been reading through the spark 1.0.0 documentation, looking at the
> script itself (spark_ec2.py), and looking at the github project
> mesos/spark-ec2.
>
> From what I can tell, the spark_ec2.py script looks up the id of the AMI
> based on the region and machine type (hvm or pvm) using static content
> derived from the github repo mesos/spark-ec2.
>
> The spark ec2 script loads the AMI id from this base url:
> https://raw.github.com/mesos/spark-ec2/v2/ami-list
> (Which presumably comes from https://github.com/mesos/spark-ec2 )
>
> For instance, I'm working with us-east-1 and pvm, I'd end up with AMI id:
> ami-5bb18832
>
> Is there a list of instructions for how this AMI was created? Assuming
> I'm starting with my own Amazon Linux image, what would I need to do to
> make it usable where I could pass that AMI id to spark_ec2.py rather than
> using the default spark-provided AMI?
>
> Thanks,
> Matt
>
