Hi Tim, Yes we are running Spark on Mesos in cluster mode with supervise flag. Submit script looks like this:
spark-submit \ --conf "spark.executor.extraJavaOptions=-XX:+UseG1GC -XX:+UseCompressedOops -XX:-UseGCOverheadLimit" \ --supervise \ --deploy-mode cluster \ --class <fullClassName> \ --master mesos://<sparkDispatcherIP>:7077 <pathToJar> Mesos version = 0.26.0 Spark version = 1.5.2 thanks, Duc On Sat, Jan 30, 2016 at 9:48 AM, Tim Chen <t...@mesosphere.io> wrote: > Hi Duc, > > Are you running Spark on Mesos with cluster mode? And what's your cluster > mode submission, and version of Spark are you running? > > Tim > > On Sat, Jan 30, 2016 at 8:19 AM, PhuDuc Nguyen <duc.was.h...@gmail.com> > wrote: > >> I have a spark job running on Mesos in multi-master and supervise mode. >> If I kill it, it is resilient as expected and respawns on another node. >> However, I cannot kill it when I need to. I have tried 2 methods: >> >> 1) ./bin/spark-class org.apache.spark.deploy.Client kill >> <masterIp:port???> <driverID> >> >> 2) ./bin/spark-submit --master mesos://<masterIp:7077> --kill <driverID> >> >> Method 2, accepts the kill request but is respawned on another node. >> Ultimately, I can't get either method to kill the job. I suspect I have >> the wrong port for the master URL during the kill request for method 1? >> I've tried every combination of IP and port I can think of, is there one I >> am missing? >> >> Ports I've tried: >> 5050 = mesos UI >> 8080 = marathon >> 7077 = spark dispatcher >> 8081 = spark drivers UI >> 4040 = spark job UI >> >> thanks, >> Duc >> > >
