Thanks for the info and good luck with 1.0. Regards, Art
On Fri, Apr 25, 2014 at 9:48 AM, Andrew Or <and...@databricks.com> wrote: > Hi Art, > > First of all thanks a lot for your PRs. We are currently in the middle of > all the Spark 1.0 release so most of us are swamped with the more core > features. To answer your questions: > > 1. Neither. We welcome changes from developers for all components of Spark, > including the EC2 scripts. Once the release is out we will have more time > to review the many PRs that we missed on the ride. > > 2. We prefer to keep the EC2 scripts within Spark, at least for now. > > Cheers, > Andrew > > On Friday, April 25, 2014, Art Peel <found...@gmail.com> wrote: > > > I've been setting up Spark cluster on EC2 using the provided > > ec2/spark_ec2.py script and am very happy I didn't have to write it from > > scratch. Thanks for providing it. > > > > There have been some issues, though, and I have had to make some > additions. > > So far, they are all additions of command-line options. For example, > the > > original script allows access from anywhere to the various ports. I've > > added an option to specify what net/mask should be allowed to access > those > > ports. > > > > I've filed a couple of pull requests, but they are not going anywhere. > > Given what I've seen of the traffic on this list, I don't feel that a > lot > > of the developers are thinking about EC2 setup. I totally agree that it > is > > not as important as improving the guts of Spark itself; nevertheless, I > > feel that being able to run Spark on EC2 smartly and easily is valuable. > > > > So, I have 2 questions for the committers: > > > > 1. Is ec2/spark_ec2.py something the committers > > a. are not thinking about? > > b. are planning to replace? > > c. other > > > > 2. Should I just start a new project based on ec2/spark_ec2.py but > without > > all the other stuff and make (and share) my changes there? > > > > Regards, > > > > Art > > >
