Hey, so currently it doesn't work because of <https://github.com/apache/spark/pull/6#issuecomment-36343187>
IntelliJ reports a lot of warnings with default settings and I haven't found a way to tell IntellJ to use different Hadoop versions yet. mvn clean compile -Pyarn fails as well (compilation errror Your command works indeed. Default yarn version is 0.23.7 which doesn't seem to work with the default 2.2.0 Hadoop version (anymore?) I was basically trying to follow the documentation: <http://spark.incubator.apache.org/docs/latest/building-with-maven.html> mvn clean compile -Pyarn-alpha -Dhadoop.version=2.0.0-cdh4.5.0 -Dyarn.version=2.0.0-cdh4.5.0 fails as well as does mvn clean compile -Pyarn-alpha Thanks for showing me a configuration that works. Unfortunately the default ones and at least one of the documented ones fail. Cheers, Lars On Fri, Feb 28, 2014 at 3:05 PM, Tom Graves <tgraves...@yahoo.com> wrote: > what build command are you using? What do you mean when you say YARN > branch? > > The yarn builds have been working fine for me with maven. Build command I > use against hadoop 2.2 or higher: mvn -Dyarn.version=2.2.0 > -Dhadoop.version=2.2.0 -Pyarn clean package -DskipTests > > Tom > > > > On Friday, February 28, 2014 6:14 AM, Lars Francke <lars.fran...@gmail.com> > wrote: > > Hey, > > I'm trying to dig into Spark's code but am running into a couple of problems. > > 1) The yarn-common directory is not included in the Maven build > causing things to fail because the dependency is missing. If I see the > history correct it used to be a Maven module but is not anymore. > > 2) When I try to include the yarn-common directory in the build things > start going bad. Compilation failures all over the place and I think > there are some dependency issues in there as well. > > This leads me to believe that either the Maven build system isn't > maintained for YARN or the whole YARN branch isn't. What's the status > here? > > Without YARN things build fine for me using Maven. > > Thanks for your help. > > Cheers, > Lars
