I'm not sure what "on the driver" means but I've tried setting spark.files.userClassPathFirst to true, in $SPARK-HOME/conf/spark-defaults.conf and also in the SparkConf programmatically; it appears to be ignored. The solution was to follow Emre's recommendation and downgrade the selected Solrj distro to 4.0.0. That did the trick as it appears to be using the same HttpClient as one used by Spark/Hadoop.
The Spark program I'm running is a jar I submit via a spark-submit invokation. On Wed, Feb 18, 2015 at 1:57 PM, Marcelo Vanzin <[email protected]> wrote: > Hello, > > On Tue, Feb 17, 2015 at 8:53 PM, dgoldenberg <[email protected]> > wrote: > > I've tried setting spark.files.userClassPathFirst to true in SparkConf > in my > > program, also setting it to true in > $SPARK-HOME/conf/spark-defaults.conf as > > Is the code in question running on the driver or in some executor? > spark.files.userClassPathFirst only applies to executors. To override > classes in the driver's classpath, you need to modify > spark.driver.extraClassPath (or --driver-class-path in spark-submit's > command line). > > In 1.3 there's an option similar to spark.files.userClassPathFirst > that works for the driver too. > > -- > Marcelo >
