Hi JM Below are my commands, tried two cases under same source code folder: a) compile with snappy parameters(failed), b) compile without snappy parameters (successful).
Regards Arthur wget http://mirrors.devlib.org/apache/hbase/stable/hbase-0.98.4-src.tar.gz tar -vxf hbase-0.98.4-src.tar.gz mv hbase-0.98.4 hbase-0.98.4-src_snappy cd hbase-0.98.4-src_snappy nano dev-support/generate-hadoopX-poms.sh (change hbase_home=“/usr/local/hadoop/hbase-0.98.4-src_snappy”) bash -x ./dev-support/generate-hadoopX-poms.sh 0.98.4 0.98.4-hadoop2 a) with snappy parameters mvn -f pom.xml.hadoop2 install -DskipTests assembly:single -Prelease,hadoop-snappy -Dhadoop-snappy.version=0.0.1-SNAPSHOT [INFO] ------------------------------------------------------------------------ [INFO] Building HBase - Server 0.98.4-hadoop2 [INFO] ------------------------------------------------------------------------ [WARNING] The POM for org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT is missing, no dependency information available [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] HBase ............................................. SUCCESS [8.192s] [INFO] HBase - Common .................................... SUCCESS [5.638s] [INFO] HBase - Protocol .................................. SUCCESS [1.535s] [INFO] HBase - Client .................................... SUCCESS [1.206s] [INFO] HBase - Hadoop Compatibility ...................... SUCCESS [0.193s] [INFO] HBase - Hadoop Two Compatibility .................. SUCCESS [0.798s] [INFO] HBase - Prefix Tree ............................... SUCCESS [0.438s] [INFO] HBase - Server .................................... FAILURE [0.234s] [INFO] HBase - Testing Util .............................. SKIPPED [INFO] HBase - Thrift .................................... SKIPPED [INFO] HBase - Shell ..................................... SKIPPED [INFO] HBase - Integration Tests ......................... SKIPPED [INFO] HBase - Examples .................................. SKIPPED [INFO] HBase - Assembly .................................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 19.474s [INFO] Finished at: Tue Aug 26 21:21:13 HKT 2014 [INFO] Final Memory: 51M/1100M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal on project hbase-server: Could not resolve dependencies for project org.apache.hbase:hbase-server:jar:0.98.4-hadoop2: Failure to find org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT in http://maven.oschina.net/content/groups/public/ was cached in the local repository, resolution will not be reattempted until the update interval of nexus-osc has elapsed or updates are forced -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :hbase-server b) try again, without snappy parameters mvn -f pom.xml.hadoop2 install -DskipTests assembly:single -Prelease [INFO] Building tar: /edh/hadoop_all_sources/hbase-0.98.4-src_snappy/hbase-assembly/target/hbase-0.98.4-hadoop2-bin.tar.gz [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] HBase ............................................. SUCCESS [3.290s] [INFO] HBase - Common .................................... SUCCESS [3.119s] [INFO] HBase - Protocol .................................. SUCCESS [0.972s] [INFO] HBase - Client .................................... SUCCESS [0.920s] [INFO] HBase - Hadoop Compatibility ...................... SUCCESS [0.167s] [INFO] HBase - Hadoop Two Compatibility .................. SUCCESS [0.504s] [INFO] HBase - Prefix Tree ............................... SUCCESS [0.382s] [INFO] HBase - Server .................................... SUCCESS [4.790s] [INFO] HBase - Testing Util .............................. SUCCESS [0.598s] [INFO] HBase - Thrift .................................... SUCCESS [1.536s] [INFO] HBase - Shell ..................................... SUCCESS [0.369s] [INFO] HBase - Integration Tests ......................... SUCCESS [0.443s] [INFO] HBase - Examples .................................. SUCCESS [0.459s] [INFO] HBase - Assembly .................................. SUCCESS [13.240s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 31.408s [INFO] Finished at: Tue Aug 26 21:22:50 HKT 2014 [INFO] Final Memory: 57M/1627M [INFO] ------------------------------------------------------------------------ On 26 Aug, 2014, at 8:52 pm, Jean-Marc Spaggiari <[email protected]> wrote: > Hi Arthur, > > How have you extracted HBase source and what command do you run to build? I > will do the same here locally so I can provide you the exact step to > complete. > > JM > > > 2014-08-26 8:42 GMT-04:00 [email protected] <[email protected] >> : > >> Hi JM >> >> Not too sure what you mean, do you mean I should create a new folder in my >> HBASE_SRC named lib/native/Linux-x86 and copy these files to this folder >> then try to compile it again? >> >> Regards >> ARthur >> >> >> On 26 Aug, 2014, at 8:17 pm, Jean-Marc Spaggiari <[email protected]> >> wrote: >> >>> Hi Arthur, >>> >>> Almost done! You now need to copy them on the HBase folder. >>> >>> hbase@hbasetest1:~/hbase-0.98.2-hadoop2/lib$ tree | grep -v .jar | grep >> -v >>> .rb >>> . >>> ├── native >>> │ └── Linux-x86 >>> │ ├── libsnappy.a >>> │ ├── libsnappy.la >>> │ ├── libsnappy.so >>> │ ├── libsnappy.so.1 >>> │ └── libsnappy.so.1.2.0 >>> >>> I don't have any hadoop-snappy lib in my hbase folder and it works very >>> well with Snappy for me... >>> >>> JM >>> >>> 2014-08-26 8:09 GMT-04:00 [email protected] < >> [email protected] >>>> : >>> >>>> Hi JM, >>>> >>>> Below are my steps to install snappy lib, do I miss something? >>>> >>>> Regards >>>> Arthur >>>> >>>> wget https://snappy.googlecode.com/files/snappy-1.1.1.tar.gz >>>> tar -vxf snappy-1.1.1.tar.gz >>>> cd snappy-1.1.1 >>>> ./configure >>>> make >>>> make install >>>> make[1]: Entering directory >> `/edh/hadoop_all_sources/snappy-1.1.1' >>>> test -z "/usr/local/lib" || /bin/mkdir -p "/usr/local/lib" >>>> /bin/sh ./libtool --mode=install /usr/bin/install -c >>>> libsnappy.la '/usr/local/lib' >>>> libtool: install: /usr/bin/install -c .libs/libsnappy.so.1.2.0 >>>> /usr/local/lib/libsnappy.so.1.2.0 >>>> libtool: install: (cd /usr/local/lib && { ln -s -f >>>> libsnappy.so.1.2.0 libsnappy.so.1 || { rm -f libsnappy.so.1 && ln -s >>>> libsnappy.so.1.2.0 libsnappy.so.1; }; }) >>>> libtool: install: (cd /usr/local/lib && { ln -s -f >>>> libsnappy.so.1.2.0 libsnappy.so || { rm -f libsnappy.so && ln -s >>>> libsnappy.so.1.2.0 libsnappy.so; }; }) >>>> libtool: install: /usr/bin/install -c .libs/libsnappy.lai >>>> /usr/local/lib/libsnappy.la >>>> libtool: install: /usr/bin/install -c .libs/libsnappy.a >>>> /usr/local/lib/libsnappy.a >>>> libtool: install: chmod 644 /usr/local/lib/libsnappy.a >>>> libtool: install: ranlib /usr/local/lib/libsnappy.a >>>> libtool: finish: >>>> >> PATH="/edh/hadoop/spark/bin:/edh/hadoop/hbase/bin:/edh/hadoop/zookeeper//bin:/edh/hadoop/yarn/hadoop/bin:/edh/hadoop/yarn/hadoop/sbin:/usr/lib64/qt-3.3/bin:/opt/apache-maven-3.1.1/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/edh/hadoop/zookeeper//bin:/edh/hadoop/hive//bin:/usr/lib/jvm/jdk1.6.0_45//bin:/root/bin:/sbin" >>>> ldconfig -n /usr/local/lib >>>> >>>> ---------------------------------------------------------------------- >>>> Libraries have been installed in: >>>> /usr/local/lib >>>> If you ever happen to want to link against installed libraries >>>> in a given directory, LIBDIR, you must either use libtool, and >>>> specify the full pathname of the library, or use the `-LLIBDIR' >>>> flag during linking and do at least one of the following: >>>> - add LIBDIR to the `LD_LIBRARY_PATH' environment variable >>>> during execution >>>> - add LIBDIR to the `LD_RUN_PATH' environment variable >>>> during linking >>>> - use the `-Wl,-rpath -Wl,LIBDIR' linker flag >>>> - have your system administrator add LIBDIR to `/etc/ld.so.conf' >>>> See any operating system documentation about shared libraries for >>>> more information, such as the ld(1) and ld.so(8) manual pages. >>>> >>>> ---------------------------------------------------------------------- >>>> test -z "/usr/local/share/doc/snappy" || /bin/mkdir -p >>>> "/usr/local/share/doc/snappy" >>>> /usr/bin/install -c -m 644 ChangeLog COPYING INSTALL NEWS README >>>> format_description.txt framing_format.txt '/usr/local/share/doc/snappy' >>>> test -z "/usr/local/include" || /bin/mkdir -p >> "/usr/local/include" >>>> /usr/bin/install -c -m 644 snappy.h snappy-sinksource.h >>>> snappy-stubs-public.h snappy-c.h '/usr/local/include' >>>> make[1]: Leaving directory `/edh/hadoop_all_sources/snappy-1.1.1' >>>> >>>> ll /usr/local/lib >>>> -rw-r--r--. 1 root root 233554 Aug 20 00:14 libsnappy.a >>>> -rwxr-xr-x. 1 root root 953 Aug 20 00:14 libsnappy.la >>>> lrwxrwxrwx. 1 root root 18 Aug 20 00:14 libsnappy.so -> >>>> libsnappy.so.1.2.0 >>>> lrwxrwxrwx. 1 root root 18 Aug 20 00:14 libsnappy.so.1 -> >>>> libsnappy.so.1.2.0 >>>> -rwxr-xr-x. 1 root root 147726 Aug 20 00:14 libsnappy.so.1.2.0 >>>> >>>> >>>> >>>> On 26 Aug, 2014, at 7:38 pm, Jean-Marc Spaggiari < >> [email protected]> >>>> wrote: >>>> >>>>> Hi Arthur, >>>>> >>>>> Do you have snappy libs installed and configured? HBase doesn't come >> with >>>>> Snappy. So yo need to have it first. >>>>> >>>>> Shameless plug: >>>>> >>>> >> http://www.spaggiari.org/index.php/hbase/how-to-install-snappy-with-1#.U_xxSqdZuZY >>>>> >>>>> This is for 0.96 but should be very similar for 0.98. I will try it >> soon >>>>> and post and update, but keep us posted here so we can support you... >>>>> >>>>> JM >>>>> >>>>> >>>>> 2014-08-26 7:34 GMT-04:00 [email protected] < >>>> [email protected] >>>>>> : >>>>> >>>>>> Hi, >>>>>> >>>>>> I need to install snappy to HBase 0.98.4. (my Hadoop version is >> 2.4.1) >>>>>> >>>>>> Can you please advise what would be wrong? Should my pom.xml be >>>> incorrect >>>>>> and missing something? >>>>>> >>>>>> Regards >>>>>> Arthur >>>>>> >>>>>> >>>>>> Below are my commands: >>>>>> bash -x ./dev-support/generate-hadoopX-poms.sh 0.98.4 0.98.4-hadoop2 >>>>>> mvn -f pom.xml.hadoop2 install -DskipTests assembly:single >>>>>> -Prelease,hadoop-snappy >>>>>> >>>>>> Iog: >>>>>> [INFO] >>>>>> >> ------------------------------------------------------------------------ >>>>>> [INFO] Building HBase - Server 0.98.4-hadoop2 >>>>>> [INFO] >>>>>> >> ------------------------------------------------------------------------ >>>>>> [WARNING] The POM for >> org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT >>>>>> is missing, no dependency information available >>>>>> [INFO] >>>>>> >> ------------------------------------------------------------------------ >>>>>> [INFO] Reactor Summary: >>>>>> [INFO] >>>>>> [INFO] HBase ............................................. SUCCESS >>>> [3.129s] >>>>>> [INFO] HBase - Common .................................... SUCCESS >>>> [3.105s] >>>>>> [INFO] HBase - Protocol .................................. SUCCESS >>>> [0.976s] >>>>>> [INFO] HBase - Client .................................... SUCCESS >>>> [0.925s] >>>>>> [INFO] HBase - Hadoop Compatibility ...................... SUCCESS >>>> [0.183s] >>>>>> [INFO] HBase - Hadoop Two Compatibility .................. SUCCESS >>>> [0.497s] >>>>>> [INFO] HBase - Prefix Tree ............................... SUCCESS >>>> [0.407s] >>>>>> [INFO] HBase - Server .................................... FAILURE >>>> [0.103s] >>>>>> [INFO] HBase - Testing Util .............................. SKIPPED >>>>>> [INFO] HBase - Thrift .................................... SKIPPED >>>>>> [INFO] HBase - Shell ..................................... SKIPPED >>>>>> [INFO] HBase - Integration Tests ......................... SKIPPED >>>>>> [INFO] HBase - Examples .................................. SKIPPED >>>>>> [INFO] HBase - Assembly .................................. SKIPPED >>>>>> [INFO] >>>>>> >> ------------------------------------------------------------------------ >>>>>> [INFO] BUILD FAILURE >>>>>> [INFO] >>>>>> >> ------------------------------------------------------------------------ >>>>>> [INFO] Total time: 9.939s >>>>>> [INFO] Finished at: Tue Aug 26 19:23:14 HKT 2014 >>>>>> [INFO] Final Memory: 61M/2921M >>>>>> [INFO] >>>>>> >> ------------------------------------------------------------------------ >>>>>> [ERROR] Failed to execute goal on project hbase-server: Could not >>>> resolve >>>>>> dependencies for project >>>> org.apache.hbase:hbase-server:jar:0.98.4-hadoop2: >>>>>> Failure to find org.apache.hadoop:hadoop-snappy:jar:0.0.1-SNAPSHOT in >>>>>> http://maven.oschina.net/content/groups/public/ was cached in the >> local >>>>>> repository, resolution will not be reattempted until the update >>>> interval of >>>>>> nexus-osc has elapsed or updates are forced -> [Help 1] >>>>>> [ERROR] >>>>>> [ERROR] To see the full stack trace of the errors, re-run Maven with >> the >>>>>> -e switch. >>>>>> [ERROR] Re-run Maven using the -X switch to enable full debug logging. >>>>>> [ERROR] >>>>>> [ERROR] For more information about the errors and possible solutions, >>>>>> please read the following articles: >>>>>> [ERROR] [Help 1] >>>>>> >>>> >> http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException >>>>>> [ERROR] >>>>>> [ERROR] After correcting the problems, you can resume the build with >> the >>>>>> command >>>>>> [ERROR] mvn <goals> -rf :hbase-server >>>>>> >>>>>> >>>> >>>> >> >>
