Hi Ye, I think git blame shows me because I fixed the formatting in core/pom.xml, but I don't actually know the original reason for setting SPARK_CLASSPATH there.
Do the tests run OK if you take it out? -Sandy On Thu, Sep 25, 2014 at 1:59 AM, Ye Xianjin <advance...@gmail.com> wrote: > hi, Sandy Ryza: > I believe It's you originally added the SPARK_CLASSPATH in > core/pom.xml in the org.scalatest section. Does this still needed in 1.1? > I noticed this setting because when I looked into the unit-tests.log, > It shows something below: > > 14/09/24 23:57:19.246 WARN SparkConf: > > SPARK_CLASSPATH was detected (set to 'null'). > > This is deprecated in Spark 1.0+. > > > > Please instead use: > > - ./spark-submit with --driver-class-path to augment the driver > classpath > > - spark.executor.extraClassPath to augment the executor classpath > > > > 14/09/24 23:57:19.246 WARN SparkConf: Setting > 'spark.executor.extraClassPath' to 'null' as a work-around. > > 14/09/24 23:57:19.247 WARN SparkConf: Setting > 'spark.driver.extraClassPath' to 'null' as a work-around. > > However I didn't set SPARK_CLASSPATH env variable. And looked into the > SparkConf.scala, If user actually set extraClassPath, the SparkConf will > throw SparkException. > -- > Ye Xianjin > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > On Tuesday, September 23, 2014 at 12:56 AM, Ye Xianjin wrote: > > > Hi: > > I notice the scalatest-maven-plugin set SPARK_CLASSPATH environment > variable for testing. But in the SparkConf.scala, this is deprecated in > Spark 1.0+. > > So what this variable for? should we just remove this variable? > > > > > > -- > > Ye Xianjin > > Sent with Sparrow (http://www.sparrowmailapp.com/?sig) > > > >
