Re: [VOTE] Apache Spark 2.1.1 (RC2)

2017-04-15 Thread Sean Owen
I don't think this is an example of Java 8 javadoc being more strict; it is not finding classes, not complaining about syntax. (Hyukjin cleaned up all of the javadoc 8 errors in master, and they're different and much more extensive!) It wouldn't necessarily break anything to build with Java 8 beca

Feedback on JavaPairRDD API

2017-04-15 Thread Matthias Boehm
After using Spark for many years, including SystemML's Spark backend, I'd like to give some feedback on potential PairRDD API extensions that I would find very useful: 1) MapToPair with preservesPartitioning flag: For many binary operations with broadcasts, we always need to use mapPartitionsToPai