On 22/10/11 06:30, Tim Broberg wrote:
I'd like to add a core module to hadoop, but I'm running into some issues
getting started.
What I want is to be able to add a native library and codec to some stable
build of hadoop, build, debug, experiment, and benchmark.
Currently, I'm trying to rebuild the Cloudera rpms so I can get a complete
stable set of source to start from. (When I tried working from the SVN trunk,
it seemed there was so much active development going on, it was hard to get
something stable to compile.)
try checking out
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20-security
for the 0.20.20x branch
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23 for
the forthcoming 0.23 release. versions 0.23+ are built with Maven, which
makes some things easier (IDE setup), other things worse
So, I'm working from the Cloudera instructions -
https://ccp.cloudera.com/display/CDHDOC/Building+RPMs+from+CDH+Source+RPMs.
I downloaded hadoop-0.20-0.20.2+923.97-1.src.rpm, installed jdk, ant, maven,
and set various environment variables:
export PATH=$PATH:/usr/local/apache-maven-3.0.3/bin
export JAVA_HOME=/usr/java/jdk1.7.0
export HADOOP_HOME=/usr/lib/hadoop-0.20
export HADOOP_VERSION=`hadoop version | head -n 1 | cut -f 2 -d " "`
I wasn't sure what to do about ANT_HOME, FORREST_HOME, or JAVA5_HOME as forrest
wasn't requested to be installed, ant doesn't appear to have a special
directory anywhere, and just generally not sure what's up with JAVA5_HOME, but
this appears to be forrest related? None of these is generating complaints when
I build.
ANT_HOME is the directory into which you installed Ant, see the ant
installation instructions
JAVA5_HOME is probably used by Forrest to create the documentation that
-if you don't ask for- won't need that or FORREST_HOME
When I build, I get a whole bunch of output including the following:
rpmbuild --rebuild $SRPM
.........<snip>............
compile:
[echo] contrib: gridmix
[javac] Compiling 31 source files to
/home/tbroberg/rpmbuild/BUILD/hadoop-0.20.2-cdh3u1/build/contrib/gridmix/classes
[javac]
/home/tbroberg/rpmbuild/BUILD/hadoop-0.20.2-cdh3u1/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java:396:
error: type argument ? extends T is not within bounds of type-variable E
[javac] private<T> String getEnumValues(Enum<? extends T>[] e) {
[javac] ^
[javac] where T,E are type-variables:
[javac] T extends Object declared in method<T>getEnumValues(Enum<?
extends T>[])
[javac] E extends Enum<E> declared in class Enum
[javac]
/home/tbroberg/rpmbuild/BUILD/hadoop-0.20.2-cdh3u1/src/contrib/gridmix/src/java/org/apache/hadoop/mapred/gridmix/Gridmix.java:399:
error: type argument ? extends T is not within bounds of type-variable E
[javac] for (Enum<? extends T> v : e) {
[javac] ^
[javac] where T,E are type-variables:
[javac] T extends Object declared in method<T>getEnumValues(Enum<?
extends T>[])
[javac] E extends Enum<E> declared in class Enum
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.
[javac] 2 errors
That could be a quirk of Java7 being fussier than Java6 -I've never seen
that error before. A quick search shows all mentions of it coming up
from July 2011
Questions:
1 - Is there a more appropriate environment to work from than the Cloudera
distribution for developing codecs?
check out the SVN source from svn.apache.org for the branch/version you
want to use
2 - If not, is this an ppropriate place to ask about how to build Cloudera?
I'd go to the cloudera lists
3 - Any suggestions for getting this rpm to rebuild?
4 - Any suggestions for editing the rpm so I can just wipe out gridmix
altogether?
n/a