Re: Python Spark Improvements (forked from Spark Improvement Proposals)

So i've been working on some very very early stage apache arrow integration. My current plan it to emulate some of how the R function execution works. If there are any other people working on similar efforts it would be good idea to combine efforts. I can see how much effort is involved in conv

Re: Python Spark Improvements (forked from Spark Improvement Proposals)

So for the jupyter integration pieces. I've made a simple library ( https://github.com/MaxPoint/spylon ) which allows a simpler way of creating a SparkContext (with all the parameters available to spark-submit) as well as some usability enhancements, progress