Re: Announcing the official Spark Job Server repo

Thx for answering! see inline for my thoughts (or misunderstanding ? ^^) Andy, doesn't Marathon handle fault tolerance amongst its apps? ie if > you say that N instances of an app are running, and one shuts off, > then it spins up another one no? > Yes indeed, but my wonder is about how to know h

Re: Announcing the official Spark Job Server repo

Andy, doesn't Marathon handle fault tolerance amongst its apps? ie if you say that N instances of an app are running, and one shuts off, then it spins up another one no? The tricky thing was that I was planning to use Akka Cluster to coordinate, but Mesos itself can be used to coordinate as well,

Re: Announcing the official Spark Job Server repo

Heya, That's cool you've already hacked something for this in the scripts! I have a related question, how would it work actually. I mean, to have this Job Server fault tolerant using Marathon, I would guess that it will need to be itself a Mesos framework, and able to publish its resources needs.

Re: Announcing the official Spark Job Server repo

+1, Evan et al. -- Christopher T. Nguyen Co-founder & CEO, Adatao linkedin.com/in/ctnguyen On Tue, Mar 18, 2014 at 1:51 PM, Evan Chan wrote: > Dear Spark developers, > > Ooyala is happy to announce that we have pushed our official, Spark > 0.9.0 / Scala 2.10-compatible, jo

Re: Announcing the official Spark Job Server repo

https://spark-project.atlassian.net/browse/SPARK-1283 On Wed, Mar 19, 2014 at 10:59 AM, Gerard Maas wrote: > this is cool +1 > > > On Wed, Mar 19, 2014 at 6:54 PM, Patrick Wendell wrote: > >> Evan - yep definitely open a JIRA. It would be nice to have a contrib >> repo set-up for the 1.0 release

Re: Announcing the official Spark Job Server repo

this is cool +1 On Wed, Mar 19, 2014 at 6:54 PM, Patrick Wendell wrote: > Evan - yep definitely open a JIRA. It would be nice to have a contrib > repo set-up for the 1.0 release. > > On Tue, Mar 18, 2014 at 11:28 PM, Evan Chan wrote: > > Matei, > > > > Maybe it's time to explore the spark-cont

Re: Announcing the official Spark Job Server repo

Evan - yep definitely open a JIRA. It would be nice to have a contrib repo set-up for the 1.0 release. On Tue, Mar 18, 2014 at 11:28 PM, Evan Chan wrote: > Matei, > > Maybe it's time to explore the spark-contrib idea again? Should I > start a JIRA ticket? > > -Evan > > > On Tue, Mar 18, 2014 at

Re: Announcing the official Spark Job Server repo

Matei, Maybe it's time to explore the spark-contrib idea again? Should I start a JIRA ticket? -Evan On Tue, Mar 18, 2014 at 4:04 PM, Matei Zaharia wrote: > Cool, glad to see this posted! I've added a link to it at > https://cwiki.apache.org/confluence/display/SPARK/Powered+By+Spark. > > Mat

Re: Announcing the official Spark Job Server repo

Andy, Yeah, we've thought of deploying this on Marathon ourselves, but we're not sure how much Mesos we're going to use yet. (Indeed if you look at bin/server_start.sh, I think I set up the PORT environment var specifically for Marathon.)This is also why we have deploy scripts which package

Re: Announcing the official Spark Job Server repo

Cool, glad to see this posted! I’ve added a link to it at https://cwiki.apache.org/confluence/display/SPARK/Powered+By+Spark. Matei On Mar 18, 2014, at 1:51 PM, Evan Chan wrote: > Dear Spark developers, > > Ooyala is happy to announce that we have pushed our official, Spark > 0.9.0 / Scala 2.

Re: Announcing the official Spark Job Server repo

tad! That's awesome. A quick question, does someone has insights regarding having such JobServers deployed using Marathon on Mesos? I'm thinking about an arch where Marathon would deploy and keep the Job Servers running along with part of the whole set of apps deployed on it regarding the res

Re: Announcing the official Spark Job Server repo

+1 On Tue, Mar 18, 2014 at 3:39 PM, Henry Saputra wrote: > W00t! > > Thanks for releasing this, Evan. > > - Henry > > On Tue, Mar 18, 2014 at 1:51 PM, Evan Chan wrote: > > Dear Spark developers, > > > > Ooyala is happy to announce that we have pushed our official, Spark > > 0.9.0 / Scala 2.10-c

Re: Announcing the official Spark Job Server repo

W00t! Thanks for releasing this, Evan. - Henry On Tue, Mar 18, 2014 at 1:51 PM, Evan Chan wrote: > Dear Spark developers, > > Ooyala is happy to announce that we have pushed our official, Spark > 0.9.0 / Scala 2.10-compatible, job server as a github repo: > > https://github.com/ooyala/spark-job

Announcing the official Spark Job Server repo

Dear Spark developers, Ooyala is happy to announce that we have pushed our official, Spark 0.9.0 / Scala 2.10-compatible, job server as a github repo: https://github.com/ooyala/spark-jobserver Complete with unit tests, deploy scripts, and examples. The original PR (#222) on incubator-spark is n