We are migrating our scripts to r3. Thr lineage is in spark-ec2 would be happy to migrate those too. Having trouble with ganglia setup currently :) Regards Mayur On 31 May 2014 09:07, "Patrick Wendell" <pwend...@gmail.com> wrote:
> Hi Jeremy, > > That's interesting, I don't think anyone has ever reported an issue > running these scripts due to Python incompatibility, but they may require > Python 2.7+. I regularly run them from the AWS Ubuntu 12.04 AMI... that > might be a good place to start. But if there is a straightforward way to > make them compatible with 2.6 we should do that. > > For r3.large, we can add that to the script. It's a newer type. Any > interest in contributing this? > > - Patrick > > On May 30, 2014 5:08 AM, "Jeremy Lee" <unorthodox.engine...@gmail.com> > wrote: > >> >> Hi there! I'm relatively new to the list, so sorry if this is a repeat: >> >> I just wanted to mention there are still problems with the EC2 scripts. >> Basically, they don't work. >> >> First, if you run the scripts on Amazon's own suggested version of linux, >> they break because amazon installs Python2.6.9, and the scripts use a >> couple of Python2.7 commands. I have to "sudo yum install python27", and >> then edit the spark-ec2 shell script to use that specific version. >> Annoying, but minor. >> >> (the base "python" command isn't upgraded to 2.7 on many systems, >> apparently because it would break yum) >> >> The second minor problem is that the script doesn't know about the >> "r3.large" servers... also easily fixed by adding to the spark_ec2.py >> script. Minor, >> >> The big problem is that after the EC2 cluster is provisioned, installed, >> set up, and everything, it fails to start up the webserver on the master. >> Here's the tail of the log: >> >> Starting GANGLIA gmond: [ OK ] >> Shutting down GANGLIA gmond: [FAILED] >> Starting GANGLIA gmond: [ OK ] >> Connection to ec2-54-183-82-48.us-west-1.compute.amazonaws.com closed. >> Shutting down GANGLIA gmond: [FAILED] >> Starting GANGLIA gmond: [ OK ] >> Connection to ec2-54-183-82-24.us-west-1.compute.amazonaws.com closed. >> Shutting down GANGLIA gmetad: [FAILED] >> Starting GANGLIA gmetad: [ OK ] >> Stopping httpd: [FAILED] >> Starting httpd: httpd: Syntax error on line 153 of >> /etc/httpd/conf/httpd.conf: Cannot load modules/mod_authn_alias.so into >> server: /etc/httpd/modules/mod_authn_alias.so: cannot open shared object >> file: No such file or directory >> [FAILED] >> >> Basically, the AMI you have chosen does not seem to have a "full" install >> of apache, and is missing several modules that are referred to in the >> httpd.conf file that is installed. The full list of missing modules is: >> >> authn_alias_module modules/mod_authn_alias.so >> authn_default_module modules/mod_authn_default.so >> authz_default_module modules/mod_authz_default.so >> ldap_module modules/mod_ldap.so >> authnz_ldap_module modules/mod_authnz_ldap.so >> disk_cache_module modules/mod_disk_cache.so >> >> Alas, even if these modules are commented out, the server still fails to >> start. >> >> root@ip-172-31-11-193 ~]$ service httpd start >> Starting httpd: AH00534: httpd: Configuration error: No MPM loaded. >> >> That means Spark 1.0.0 clusters on EC2 are Dead-On-Arrival when run >> according to the instructions. Sorry. >> >> Any suggestions on how to proceed? I'll keep trying to fix the webserver, >> but (a) changes to httpd.conf get blown away by "resume", and (b) anything >> I do has to be redone every time I provision another cluster. Ugh. >> >> -- >> Jeremy Lee BCompSci(Hons) >> The Unorthodox Engineers >> > > > > >
