Re: Dead lock running multiple Spark jobs on Mesos

2014-05-16 Thread Martin Weindel
ing two jobs concurrently. Alternatively, switching to fine-grained mode would help here too at the expense of higher latency on startup. On Mon, May 12, 2014 at 12:37 PM, Martin Weindel mailto:martin.wein...@gmail.com>> wrote: I'm using a current Spark 1.0.0-SNAPSHOT

Dead lock running multiple Spark Jobs on Mesos

2014-05-13 Thread Martin Weindel
I'm using a current Spark 1.0.0-SNAPSHOT for Hadoop 2.2.0 on Mesos 0.17.0. If I run a single Spark Job, the job runs fine on Mesos. Running multiple Spark Jobs also works, if I'm using the coarse-grained mode ("spark.mesos.coarse" = true). But if I run two Spark Jobs in parallel using the fine-gr

Dead lock running multiple Spark jobs on Mesos

2014-05-12 Thread Martin Weindel
I'm using a current Spark 1.0.0-SNAPSHOT for Hadoop 2.2.0 on Mesos 0.17.0. If I run a single Spark Job, the job runs fine on Mesos. Running multiple Spark Jobs also works, if I'm using the coarse-grained mode ("spark.mesos.coarse" = true). But if I run two Spark Jobs in parallel using the fin