Re: Spark on Mesos vs Yarn

2015-05-27 Thread Bharath Ravi Kumar
A follow up : considering that spark on mesos is indeed important to databricks, its partners and the community, fundamental issues like spark-6284 shouldn't be languishing for this long. A mesos cluster hosting diverse (i.e.multi-tenant) workloads is a common scenario in production for serious us

Re: Spark on Mesos vs Yarn

2015-05-15 Thread Iulian DragoČ™
Hi Ankur, Just to add a thought to Tim's excellent answer, Spark on Mesos is very important to us and is the recommended deployment for our customers as Typesafe. Thanks for pointing to your PR, I see Tim already went through a round of reviews. It seems very useful, I'll give it a try as well.

Re: Spark on Mesos vs Yarn

2015-05-15 Thread Ankur Chauhan
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Tim, Thanks for such a detailed email. I am excited to hear about the new features, I had a pull request going for adding "attribute based filtering in the mesos scheduler" but it hasn't received much love - https://github.com/apache/spark/pull/556

Re: Spark on Mesos vs Yarn

2015-05-15 Thread Tim Chen
Hi Ankur, This is a great question as I've heard similar concerns about Spark on Mesos. At the time when I started to contribute to Spark on Mesos approx half year ago, the Mesos scheduler and related code hasn't really got much attention from anyone and it was pretty much in maintenance mode. A