Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-21 Thread Tim Chen
aults.conf from somewhere, >>>>> or >>>>> at least allow the user to inform the dispatcher through spark-submit that >>>>> those properties will be available once the job starts. >>>>> >>>>> Finally, I don't think the d

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-21 Thread Alan Braithwaite
n the wrong >>>> direction. Also let me know if I should open some tickets for these >>>> issues. >>>> >>>> Thanks, >>>> - Alan >>>> >>>> On Fri, Sep 11, 2015 at 1:05 PM, Tim Chen wrote: >>>> >>>&

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-19 Thread Timothy Chen
n't think the dispatcher should crash in this event. It >>>>> seems not exceptional that a job is misconfigured when submitted. >>>>> >>>>> Please direct me on the right path if I'm headed in the wrong direction. >>>>> Also l

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-19 Thread Alan Braithwaite
. >>> >>> Thanks, >>> - Alan >>> >>> On Fri, Sep 11, 2015 at 1:05 PM, Tim Chen wrote: >>> >>>> Yes you can create an issue, or actually contribute a patch to update >>>> it :) >>>> >>>> Sorry the

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-19 Thread Tim Chen
t; On Fri, Sep 11, 2015 at 11:11 AM, Tom Waterhouse (tomwater) < >>> tomwa...@cisco.com> wrote: >>> >>>> Tim, >>>> >>>> Thank you for the explanation. You are correct, my Mesos experience is >>>> very light, and I haven’t

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-17 Thread Alan Braithwaite
ia Marathon yet. What you >>> have stated here makes sense, I will look into doing this. >>> >>> Adding this info to the docs would be great. Is the appropriate action >>> to create an issue regarding improvement of the docs? For those of us who >>> ar

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-17 Thread Timothy Chen
stated here makes sense, I will look into doing this. >>> >>> Adding this info to the docs would be great. Is the appropriate action to >>> create an issue regarding improvement of the docs? For those of us who are >>> gaining the experience having such a pointer is

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-17 Thread Alan Braithwaite
e correct, my Mesos experience is >>>> very light, and I haven’t deployed anything via Marathon yet. What you >>>> have stated here makes sense, I will look into doing this. >>>> >>>> Adding this info to the docs would be great. Is the appropriate act

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-17 Thread Alan Braithwaite
;> Adding this info to the docs would be great. Is the appropriate action >>> to create an issue regarding improvement of the docs? For those of us who >>> are gaining the experience having such a pointer is very helpful. >>> >>> Tom >>> >>>

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-17 Thread Alan Braithwaite
helpful. >> >> Tom >> >> From: Tim Chen >> Date: Thursday, September 10, 2015 at 10:25 AM >> To: Tom Waterhouse >> Cc: "user@spark.apache.org" >> Subject: Re: Spark on Mesos with Jobs in Cluster Mode Documentation >> >>

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-11 Thread Tim Chen
issue regarding improvement of the docs? For those of us who are > gaining the experience having such a pointer is very helpful. > > Tom > > From: Tim Chen > Date: Thursday, September 10, 2015 at 10:25 AM > To: Tom Waterhouse > Cc: "user@spark.apache.org"

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-11 Thread Tom Waterhouse (tomwater)
@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: Spark on Mesos with Jobs in Cluster Mode Documentation Hi Tom, Sorry the documentation isn't really rich, since it's probably assuming users understands how Mesos and framework w

Re: Spark on Mesos with Jobs in Cluster Mode Documentation

2015-09-10 Thread Tim Chen
Hi Tom, Sorry the documentation isn't really rich, since it's probably assuming users understands how Mesos and framework works. First I need explain the rationale of why create the dispatcher. If you're not familiar with Mesos yet, each node in your datacenter is installed a Mesos slave where it