Hi Art, First of all thanks a lot for your PRs. We are currently in the middle of all the Spark 1.0 release so most of us are swamped with the more core features. To answer your questions:
1. Neither. We welcome changes from developers for all components of Spark, including the EC2 scripts. Once the release is out we will have more time to review the many PRs that we missed on the ride. 2. We prefer to keep the EC2 scripts within Spark, at least for now. Cheers, Andrew On Friday, April 25, 2014, Art Peel <found...@gmail.com> wrote: > I've been setting up Spark cluster on EC2 using the provided > ec2/spark_ec2.py script and am very happy I didn't have to write it from > scratch. Thanks for providing it. > > There have been some issues, though, and I have had to make some additions. > So far, they are all additions of command-line options. For example, the > original script allows access from anywhere to the various ports. I've > added an option to specify what net/mask should be allowed to access those > ports. > > I've filed a couple of pull requests, but they are not going anywhere. > Given what I've seen of the traffic on this list, I don't feel that a lot > of the developers are thinking about EC2 setup. I totally agree that it is > not as important as improving the guts of Spark itself; nevertheless, I > feel that being able to run Spark on EC2 smartly and easily is valuable. > > So, I have 2 questions for the committers: > > 1. Is ec2/spark_ec2.py something the committers > a. are not thinking about? > b. are planning to replace? > c. other > > 2. Should I just start a new project based on ec2/spark_ec2.py but without > all the other stuff and make (and share) my changes there? > > Regards, > > Art >
