You should be able to get it to work with 2.0 as uber jar. What type cluster you are running on? YARN? And what distribution?
On Sun, Sep 4, 2016 at 8:48 PM -0700, "Holden Karau" <[email protected]<mailto:[email protected]>> wrote: You really shouldn't mix different versions of Spark between the master and worker nodes, if your going to upgrade - upgrade all of them. Otherwise you may get very confusing failures. On Monday, September 5, 2016, Rex X <[email protected]<mailto:[email protected]>> wrote: Wish to use the Pivot Table feature of data frame which is available since Spark 1.6. But the spark of current cluster is version 1.5. Can we install Spark 2.0 on the master node to work around this? Thanks! -- Cell : 425-233-8271 Twitter: https://twitter.com/holdenkarau
