Re: Updating docs for running on Mesos

Perhaps linking to a Mesos page, which then can list the various package incantations. Cheers, Tim - Original Message - > From: "Matei Zaharia" > To: dev@spark.apache.org > Sent: Tuesday, May 13, 2014 2:59:42 AM > Subject: Re: Updating docs for running on Mesos &

Re: Updating docs for running on Mesos

Great work!. I just left some comments in the PR. In summary, it would be great to have more background on how Spark works on Mesos and how the different elements interact. That will (hopefully) help understanding the practicalities of the common assembly location (http/hdfs) and how the jobs are d

Re: Updating docs for running on Mesos

Andrew, Mesosphere has binary releases here: http://mesosphere.io/downloads/ (Anecdote: I actually burned a CPU building Mesos from source. No kidding - it was coming, as the laptop was crashing from time to time, but the mesos build was that one drop too much) kr, Gerard. On Tue, May 13, 201

Re: Updating docs for running on Mesos

Completely agree about preferring to link to the upstream project rather than a company's -- the only reason I'm using mesosphere's now is that I see no alternative from mesos.apache.org I included instructions for both using Mesosphere's packages and building from scratch in the PR: https://githu

Re: Updating docs for running on Mesos

I’ll ask the Mesos folks about this. Unfortunately it might be tough to link only to a company’s builds; but we can perhaps include them in addition to instructions for building Mesos from Apache. Matei On May 12, 2014, at 11:55 PM, Gerard Maas wrote: > Andrew, > > Mesosphere has binary rele

Re: Updating docs for running on Mesos

I have a draft of my proposed changes here: https://github.com/apache/spark/pull/756 https://issues.apache.org/jira/browse/SPARK-1818 Thanks! Andrew On Mon, May 12, 2014 at 9:57 PM, Andrew Ash wrote: > As far as I know, the upstream doesn't release binaries, only source code. > The downloads

Re: Updating docs for running on Mesos

As far as I know, the upstream doesn't release binaries, only source code. The downloads page for 0.18.0 only has a source tarball. Is there a binary release somewhere from Mesos that I'm missing? On Sun, May 11, 2014 at 2:16 PM, Patrick Wendell wrote: >

Re: Updating docs for running on Mesos

For trimming the Running Alongside Hadoop section I mostly think there should be a separate Spark+HDFS section and have the CDH+HDP page be merged into that one, but I supposed that's a separate docs change. On Sun, May 11, 2014 at 4:28 PM, Andy Konwinski wrote: > Thanks for suggesting this and

Re: Updating docs for running on Mesos

Andrew, Updating these docs would be great! I think this would be a welcome change. In terms of packaging, it would be good to mention the binaries produced by the upstream project as well, in addition to Mesosphere. - Patrick On Thu, May 8, 2014 at 12:51 AM, Andrew Ash wrote: > The docs for h

Re: Updating docs for running on Mesos

Thanks for suggesting this and volunteering to do it. On May 11, 2014 3:32 AM, "Andrew Ash" wrote: > > The docs for how to run Spark on Mesos have changed very little since > 0.6.0, but setting it up is much easier now than then. Does it make sense > to revamp with the below changes? > > > You n