Ah, right. So only the launch script has changed. Everything else is still essentially binary compatible?
Well, that makes it too easy! Thanks! On Wed, Jun 18, 2014 at 2:35 PM, Patrick Wendell <pwend...@gmail.com> wrote: > Actually you'll just want to clone the 1.0 branch then use the > spark-ec2 script in there to launch your cluster. The --spark-git-repo > flag is if you want to launch with a different version of Spark on the > cluster. In your case you just need a different version of the launch > script itself, which will be present in the 1.0 branch of Spark. > > - Patrick > > On Tue, Jun 17, 2014 at 9:29 PM, Jeremy Lee > <unorthodox.engine...@gmail.com> wrote: > > I am about to spin up some new clusters, so I may give that a go... any > > special instructions for making them work? I assume I use the " > > --spark-git-repo=" option on the spark-ec2 command. Is it as easy as > > concatenating your string as the value? > > > > On cluster management GUIs... I've been looking around at Amabari, > Datastax, > > Cloudera, OpsCenter etc. Not totally convinced by any of them yet. Anyone > > using a good one I should know about? I'm really beginning to lean in the > > direction of Cassandra as the distributed data store... > > > > > > On Wed, Jun 18, 2014 at 1:46 PM, Patrick Wendell <pwend...@gmail.com> > wrote: > >> > >> By the way, in case it's not clear, I mean our maintenance branches: > >> > >> https://github.com/apache/spark/tree/branch-1.0 > >> > >> On Tue, Jun 17, 2014 at 8:35 PM, Patrick Wendell <pwend...@gmail.com> > >> wrote: > >> > Hey Jeremy, > >> > > >> > This is patched in the 1.0 and 0.9 branches of Spark. We're likely to > >> > make a 1.0.1 release soon (this patch being one of the main reasons), > >> > but if you are itching for this sooner, you can just checkout the head > >> > of branch-1.0 and you will be able to use r3.XXX instances. > >> > > >> > - Patrick > >> > > >> > On Tue, Jun 17, 2014 at 4:17 PM, Jeremy Lee > >> > <unorthodox.engine...@gmail.com> wrote: > >> >> Some people (me included) might have wondered why all our m1.large > spot > >> >> instances (in us-west-1) shut down a few hours ago... > >> >> > >> >> Simple reason: The EC2 spot price for Spark's default "m1.large" > >> >> instances > >> >> just jumped from 0.016 per hour, to about 0.750. Yes, Fifty times. > >> >> Probably > >> >> something to do with world cup. > >> >> > >> >> So far this is just us-west-1, but prices have a tendency to equalize > >> >> across > >> >> centers as the days pass. Time to make backups and plans. > >> >> > >> >> "m3" spot prices are still down at $0.02 (and being new, will be > >> >> bypassed by > >> >> older systems), so it would be REAAALLYY nice if there had been some > >> >> progress on that issue. Let me know if I can help with testing and > >> >> whatnot. > >> >> > >> >> > >> >> -- > >> >> Jeremy Lee BCompSci(Hons) > >> >> The Unorthodox Engineers > > > > > > > > > > -- > > Jeremy Lee BCompSci(Hons) > > The Unorthodox Engineers > -- Jeremy Lee BCompSci(Hons) The Unorthodox Engineers
