Re: Issues when combining Spark and a third party java library

Yes, Thank you. -- Henri Maxime Demoulin 2015-07-12 2:53 GMT-04:00 Akhil Das : > Did you try setting the HADOOP_CONF_DIR? > > Thanks > Best Regards > > On Sat, Jul 11, 2015 at 3:17 AM, maxdml wrote: > >> Also, it's worth noting that I'm using the prebuilt version for hadoop 2.4 >> and higher f

Re: Issues when combining Spark and a third party java library

Did you try setting the HADOOP_CONF_DIR? Thanks Best Regards On Sat, Jul 11, 2015 at 3:17 AM, maxdml wrote: > Also, it's worth noting that I'm using the prebuilt version for hadoop 2.4 > and higher from the official website. > > > > -- > View this message in context: > http://apache-spark-user-

Re: Issues when combining Spark and a third party java library

Also, it's worth noting that I'm using the prebuilt version for hadoop 2.4 and higher from the official website. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Issues-when-combining-Spark-and-a-third-party-java-library-tp21367p23770.html Sent from the Apach

Re: Issues when combining Spark and a third party java library

I'm using hadoop 2.5.2 with spark 1.4.0 and I can also see in my logs: 15/07/09 06:39:02 DEBUG HadoopRDD: SplitLocationInfo and other new Hadoop classes are unavailable. Using the older Hadoop location info code. java.lang.ClassNotFoundException: org.apache.hadoop.mapred.InputSplitWithLocationInfo

Re: Issues when combining Spark and a third party java library

Okay, I finally tried to change the Hadoop-client version from 2.4.0 to 2.5.2 and that mysteriously fixed everything.. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Issues-when-combining-Spark-and-a-third-party-java-library-tp21367p21387.html Sent from th

Re: Issues when combining Spark and a third party java library

To clarify: I'm currently working on this locally, running on a laptop and I do not use Spark-submit (using Eclipse to run my applications currently). I've tried running both on Mac OS X and in a VM running Ubuntu. Furthermore, I've got the VM from a fellow worker which has no issues running his Sp

Re: Issues when combining Spark and a third party java library

Its more like, Spark is not able to find the hadoop jars. Try setting the HADOOP_CONF_DIR and also make sure *-site.xml are available in the CLASSPATH/SPARK_CLASSPATH. Thanks Best Regards On Mon, Jan 26, 2015 at 7:28 PM, Staffan wrote: > I'm using Maven and Eclipse to build my project. I'm lett