Re: Re: spark 1.3.1 jars in repo1.maven.org

gt;>> * "Exception in thread "Spark Context Cleaner" >>>> java.lang.NoClassDefFoundError: 0 >>>> at >>>> org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:149)" &g

Re: Re: spark 1.3.1 jars in repo1.maven.org

k Context Cleaner" >>>> java.lang.NoClassDefFoundError: 0 >>>> at >>>> org.apache.spark.ContextCleaner$$anonfun$org$apache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:149)" >>>> * We've been having clashing

Re: Re: spark 1.3.1 jars in repo1.maven.org

ache$spark$ContextCleaner$$keepCleaning$1.apply$mcV$sp(ContextCleaner.scala:149)" >>> * We've been having clashing dependencies between a colleague and I >>> because of the aforementioned classpath issue >>> * The clashing dependencies are also causing issues with

Re: Re: spark 1.3.1 jars in repo1.maven.org

e available in the classloader from Spark and don't clash with >> existing libraries we have. >> >> More anon, >> >> Cheers, >> Edward >> >> >> >> Original Message >> Subject: Re: spark 1.3.1 jars in repo1.maven.org

Re: Re: spark 1.3.1 jars in repo1.maven.org

ailable in the classloader from Spark and don't clash with > existing libraries we have. > > More anon, > > Cheers, > Edward > > > > Original Message > Subject: Re: spark 1.3.1 jars in repo1.maven.org Date: 2015-05-20 00:38 > From: Sea

Fwd: Re: spark 1.3.1 jars in repo1.maven.org

encies are also causing issues with what jetty libraries are available in the classloader from Spark and don't clash with existing libraries we have. More anon, Cheers, Edward ---- Original Message ---- Subject: Re: spark 1.3.1 jars in repo1.maven.org Date: 2015-05-20 00:38 From: Sea

Re: spark 1.3.1 jars in repo1.maven.org

Yes, the published artifacts can only refer to one version of anything (OK, modulo publishing a large number of variants under classifiers). You aren't intended to rely on Spark's transitive dependencies for anything. Compiling against the Spark API has no relation to what version of Hadoop it bin

Re: spark 1.3.1 jars in repo1.maven.org

I think your observation is correct. e.g. http://mvnrepository.com/artifact/org.apache.spark/spark-core_2.10/1.3.1 shows that it depends on hadoop-client from hadoop 2.2 Cheers On Tue, May 19, 2015 at 6:17 PM, Edward Sargisson w