Hi all, Mesos is the only cluster manager that has a fine-grained mode, but it's more often than not problematic, and it's a maintenance burden. I'd like to suggest removing it in the 2.0 release.
A few reasons: - code/maintenance complexity. The two modes duplicate a lot of functionality (and sometimes code) that leads to subtle differences or bugs. See SPARK-10444 <https://issues.apache.org/jira/browse/SPARK-10444> and also this thread <https://mail-archives.apache.org/mod_mbox/spark-user/201510.mbox/%3ccalxmp-a+aygnwsiytm8ff20-mgwhykbhct94a2hwzth1jwh...@mail.gmail.com%3E> and MESOS-3202 <https://issues.apache.org/jira/browse/MESOS-3202> - it's not widely used (Reynold's previous thread <http://apache-spark-developers-list.1001551.n3.nabble.com/Please-reply-if-you-use-Mesos-fine-grained-mode-td14930.html> got very few responses from people relying on it) - similar functionality can be achieved with dynamic allocation + coarse-grained mode I suggest that Spark 1.6 already issues a warning if it detects fine-grained use, with removal in the 2.0 release. Thoughts? iulian
