Re: [DISCUSS][SPARK-25299] SPIP: Shuffle storage API

2019-06-05 Thread Matt Cheah
Hi everyone, I wanted to pick this back up again. The discussion has quieted down both on this thread and on the document. We made a few revisions to the document to hopefully make it easier to read and to clarify our criteria for success in the project. Some of the APIs have also been a

Re: Closing a SparkSession stops the SparkContext

2019-06-05 Thread Vinoo Ganesh
Picking up this email thread again around point #1 below, filed https://issues.apache.org/jira/browse/SPARK-27958 and put up a PR (still have to write tests) https://github.com/apache/spark/pull/24807 just to begin the conversation. From: Ryan Blue Reply-To: "rb...@netflix.com" Date: Wednesd

CI build issue with MiMa tests

2019-06-05 Thread Martin Junghanns
Hi all, While introducing new modules for Spark Graph (https://github.com/apache/spark/pull/24490), builds started to turn red because of failing MiMa (binary compatibility) tests. We had one successful build, however, all subsequent builds fail: Successful run: https://github.com/apache/sp