On Thu, Jun 26, 2014 at 1:44 PM, Robert James <srobertja...@gmail.com> wrote: > I downloaded the Spark distro for Hadoop 2, and installed it on my > machine. But the code doesn't have a reference to that path - it uses > sbt for dependencies. As far as I can tell, using sbt or maven or ivy > will always result in a transitive dependency to Hadoop 1 ????
That's what happens now, yes. There is one set of artifacts out there, and it is the Hadoop 1 version. (FWIW I also think it would help to have Hadoop 2-flavored artifacts published too.) > Shouldn't there be a maven option to use Hadoop 2, just like there is > a different package to download? You mean in your dependencies? yeah, you override the hadoop-client dependency.
