Re: new Catalyst/SQL component merged into master

2014-03-23 Thread Evan Chan
Hi Michael, Congrats, this is really neat! What thoughts do you have regarding adding indexing support and predicate pushdown to this SQL framework?Right now we have custom bitmap indexing to speed up queries, so we're really curious as far as the architectural direction. -Evan On Fri, Mar

Re: Spark 0.9.1 release

2014-03-23 Thread Evan Chan
I also have a really minor fix for SPARK-1057 (upgrading fastutil), could that also make it in? -Evan On Sun, Mar 23, 2014 at 11:01 PM, Shivaram Venkataraman wrote: > Sorry this request is coming in a bit late, but would it be possible to > backport SPARK-979[1] to branch-0.9 ? This is the pat

Re: Announcing the official Spark Job Server repo

2014-03-23 Thread Evan Chan
Andy, doesn't Marathon handle fault tolerance amongst its apps? ie if you say that N instances of an app are running, and one shuts off, then it spins up another one no? The tricky thing was that I was planning to use Akka Cluster to coordinate, but Mesos itself can be used to coordinate as well,

Re: Spark 0.9.1 release

2014-03-23 Thread Shivaram Venkataraman
Sorry this request is coming in a bit late, but would it be possible to backport SPARK-979[1] to branch-0.9 ? This is the patch for randomizing executor offers and I would like to use this in a release sooner rather than later. Thanks Shivaram [1] https://github.com/apache/spark/commit/556c56689b