On Tue, Nov 17, 2015 at 5:16 AM, Iulian Dragoș <iulian.dra...@typesafe.com> wrote:
> I think it actually tries to send all properties as part of > `SPARK_EXECUTOR_OPTS`, which may not be everything that's needed: > > > https://github.com/jayv/spark/blob/mesos_cluster_params/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala#L375-L377 > > Aha that's interesting, I overlooked that line, I'll debug some more today because I know for sure that those options don't make it onto the commandline when I was running it in my debugger. > Can you please open a Jira ticket and describe also the symptoms? This > might be related, or the same issue: SPARK-11280 > <https://issues.apache.org/jira/browse/SPARK-11280> and also SPARK-11327 > <https://issues.apache.org/jira/browse/SPARK-11327> > SPARK-11327 <https://issues.apache.org/jira/browse/SPARK-11327> is exactly my problem, but I don't run docker. - Jo On Tue, Nov 17, 2015 at 2:46 AM, Jo Voordeckers <jo.voordeck...@gmail.com> > wrote: > >> >> Hi all, >> >> I'm running the mesos cluster dispatcher, however when I submit jobs with >> things like jvm args, classpath order and UI port aren't added to the >> commandline executed by the mesos scheduler. In fact it only cares about >> the class, jar and num cores/mem. >> >> >> https://github.com/jayv/spark/blob/mesos_cluster_params/core/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosClusterScheduler.scala#L412-L424 >> >> I've made an attempt at adding a few of the args that I believe are >> useful to the MesosClusterScheduler class, which seems to solve my problem. >> >> Please have a look: >> >> https://github.com/apache/spark/pull/9752 >> >> Thanks >> >> - Jo Voordeckers >> >> > > > -- > > -- > Iulian Dragos > > ------ > Reactive Apps on the JVM > www.typesafe.com > >
