Hi, Thanks! I downloaded 1.5.0.1, if I add -DskipTests, I compiled it successfully.
However, if run the following (i.e. with tests) , I still got error. Many thanks. Regards Arthur mvn clean install -Dhadoop.profile=hbase-98 -Dhbase.version=0.98.3-hadoop2 [INFO] --- maven-install-plugin:2.3.1:install (default-install) @ flume-ng-embedded-agent --- [INFO] Installing /home/hduser/apache-flume-1.5.0.1-src/flume-ng-embedded-agent/target/flume-ng-embedded-agent-1.5.0.1.jar to /root/.m2/repository/org/apache/flume/flume-ng-embedded-agent/1.5.0.1/flume-ng-embedded-agent-1.5.0.1.jar [INFO] Installing /home/hduser/apache-flume-1.5.0.1-src/flume-ng-embedded-agent/pom.xml to /root/.m2/repository/org/apache/flume/flume-ng-embedded-agent/1.5.0.1/flume-ng-embedded-agent-1.5.0.1.pom [INFO] [INFO] ------------------------------------------------------------------------ [INFO] Building Flume NG HBase Sink 1.5.0.1 [INFO] ------------------------------------------------------------------------ [INFO] [INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @ flume-ng-hbase-sink --- [INFO] Deleting /home/hduser/apache-flume-1.5.0.1-src/flume-ng-sinks/flume-ng-hbase-sink/target [INFO] [INFO] --- maven-remote-resources-plugin:1.1:process (default) @ flume-ng-hbase-sink --- [INFO] [INFO] --- maven-resources-plugin:2.4.3:resources (default-resources) @ flume-ng-hbase-sink --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/hduser/apache-flume-1.5.0.1-src/flume-ng-sinks/flume-ng-hbase-sink/src/main/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) @ flume-ng-hbase-sink --- [INFO] Compiling 11 source files to /home/hduser/apache-flume-1.5.0.1-src/flume-ng-sinks/flume-ng-hbase-sink/target/classes [INFO] [INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) @ flume-ng-hbase-sink --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] skip non existing resourceDirectory /home/hduser/apache-flume-1.5.0.1-src/flume-ng-sinks/flume-ng-hbase-sink/src/test/resources [INFO] Copying 3 resources [INFO] [INFO] --- maven-compiler-plugin:2.3.2:testCompile (default-testCompile) @ flume-ng-hbase-sink --- [INFO] Compiling 7 source files to /home/hduser/apache-flume-1.5.0.1-src/flume-ng-sinks/flume-ng-hbase-sink/target/test-classes [INFO] [INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @ flume-ng-hbase-sink --- [INFO] Surefire report directory: /home/hduser/apache-flume-1.5.0.1-src/flume-ng-sinks/flume-ng-hbase-sink/target/surefire-reports ------------------------------------------------------- T E S T S ------------------------------------------------------- ------------------------------------------------------- T E S T S ------------------------------------------------------- Running org.apache.flume.sink.hbase.TestAsyncHBaseSink Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 35.333 sec <<< FAILURE! org.apache.flume.sink.hbase.TestAsyncHBaseSink Time elapsed: 35333 sec <<< ERROR! java.io.IOException: Shutting down at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:231) at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:93) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:890) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:854) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:771) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:742) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:711) at org.apache.flume.sink.hbase.TestAsyncHBaseSink.setUp(TestAsyncHBaseSink.java:69) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) Caused by: java.lang.RuntimeException: Master not active after 30 seconds at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:190) at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:425) at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:224) ... 29 more Running org.apache.flume.sink.hbase.TestRegexHbaseEventSerializer Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.307 sec Running org.apache.flume.sink.hbase.TestHBaseSinkCreation Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.174 sec Running org.apache.flume.sink.hbase.TestHBaseSink Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 33.782 sec <<< FAILURE! org.apache.flume.sink.hbase.TestHBaseSink Time elapsed: 33781 sec <<< ERROR! java.io.IOException: Shutting down at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:231) at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:93) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:890) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:854) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:771) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:742) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:711) at org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:81) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) Caused by: java.lang.RuntimeException: Master not active after 30 seconds at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:190) at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:425) at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:224) ... 29 more Results : Tests in error: org.apache.flume.sink.hbase.TestAsyncHBaseSink: Shutting down org.apache.flume.sink.hbase.TestHBaseSink: Shutting down Tests run: 8, Failures: 0, Errors: 2, Skipped: 0 [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] Apache Flume ...................................... SUCCESS [2.018s] [INFO] Flume NG SDK ...................................... SUCCESS [1:13.040s] [INFO] Flume NG Configuration ............................ SUCCESS [1.213s] [INFO] Flume NG Core ..................................... SUCCESS [4:48.152s] [INFO] Flume NG Sinks .................................... SUCCESS [0.053s] [INFO] Flume NG HDFS Sink ................................ SUCCESS [4:37.424s] [INFO] Flume NG IRC Sink ................................. SUCCESS [1.214s] [INFO] Flume NG Channels ................................. SUCCESS [0.045s] [INFO] Flume NG JDBC channel ............................. SUCCESS [2:17.684s] [INFO] Flume NG file-based channel ....................... SUCCESS [10:23.760s] [INFO] Flume NG Spillable Memory channel ................. SUCCESS [4:43.061s] [INFO] Flume NG Node ..................................... SUCCESS [24.352s] [INFO] Flume NG Embedded Agent ........................... SUCCESS [11.610s] [INFO] Flume NG HBase Sink ............................... FAILURE [1:13.813s] [INFO] Flume NG ElasticSearch Sink ....................... SKIPPED [INFO] Flume NG Morphline Solr Sink ...................... SKIPPED [INFO] Flume NG Kite Dataset Sink ........................ SKIPPED [INFO] Flume Sources ..................................... SKIPPED [INFO] Flume Scribe Source ............................... SKIPPED [INFO] Flume JMS Source .................................. SKIPPED [INFO] Flume Twitter Source .............................. SKIPPED [INFO] Flume legacy Sources .............................. SKIPPED [INFO] Flume legacy Avro source .......................... SKIPPED [INFO] Flume legacy Thrift Source ........................ SKIPPED [INFO] Flume NG Clients .................................. SKIPPED [INFO] Flume NG Log4j Appender ........................... SKIPPED [INFO] Flume NG Tools .................................... SKIPPED [INFO] Flume NG distribution ............................. SKIPPED [INFO] Flume NG Integration Tests ........................ SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 29:58.146s [INFO] Finished at: Sun Jul 27 16:04:02 HKT 2014 [INFO] Final Memory: 70M/556M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.3:test (default-test) on project flume-ng-hbase-sink: There are test failures. [ERROR] [ERROR] Please refer to /home/hduser/apache-flume-1.5.0.1-src/flume-ng-sinks/flume-ng-hbase-sink/target/surefire-reports for the individual test results. [ERROR] -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn <goals> -rf :flume-ng-hbase-sink Detail error log: tail flume-ng-sinks/flume-ng-hbase-sink/target/surefire-reports/org.apache.flume.sink.hbase.TestHBaseSink.txt -n 1000 ------------------------------------------------------------------------------- Test set: org.apache.flume.sink.hbase.TestHBaseSink ------------------------------------------------------------------------------- Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 33.782 sec <<< FAILURE! org.apache.flume.sink.hbase.TestHBaseSink Time elapsed: 33781 sec <<< ERROR! java.io.IOException: Shutting down at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:231) at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:93) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:890) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:854) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:771) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:742) at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:711) at org.apache.flume.sink.hbase.TestHBaseSink.setUpOnce(TestHBaseSink.java:81) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:27) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30) at org.junit.runners.ParentRunner.run(ParentRunner.java:300) at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252) at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141) at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189) at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165) at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85) at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115) at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75) Caused by: java.lang.RuntimeException: Master not active after 30 seconds at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:190) at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:425) at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:224) ... 29 more On 27 Jul, 2014, at 9:26 am, Hari Shreedharan <hshreedha...@cloudera.com> wrote: > Looks like you are trying to build flume-1.5.0. To build with hbase-98 > support, you need to either checkout trunk branch or the release-1.5.0.1 tag > (that release was specifically meant to add hbase-98 support). > > Hari > > On Sat, Jul 26, 2014 at 5:05 PM, a...@hsk.hk <a...@hsk.hk> wrote: > Hi, > > Tried: > > mvn clean install -Dhadoop.profile=hbase-98 -Dhbase.version=0.98.3 > [INFO] Scanning for projects... > [ERROR] The build could not read 3 projects -> [Help 1] > [ERROR] > [ERROR] The project org.apache.flume.flume-ng-sinks:flume-hdfs-sink:1.5.0 > (flume-ng-sinks/flume-hdfs-sink/pom.xml) has 2 errors > [ERROR] 'dependencies.dependency.artifactId' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar with value > '${hadoop.common.artifact.id}' does not match a valid id pattern. @ line 89, > column 19 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar is missing. @ line 87, > column 17 > [ERROR] > [ERROR] The project > org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:1.5.0 > (flume-ng-sinks/flume-ng-hbase-sink/pom.xml) has 2 errors > [ERROR] 'dependencies.dependency.artifactId' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar with value > '${hadoop.common.artifact.id}' does not match a valid id pattern. @ line 96, > column 19 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar is missing. @ line 94, > column 17 > [ERROR] > [ERROR] The project org.apache.flume:flume-ng-tests:1.5.0 > (flume-ng-tests/pom.xml) has 2 errors > [ERROR] 'dependencies.dependency.artifactId' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar with value > '${hadoop.common.artifact.id}' does not match a valid id pattern. @ line 60, > column 19 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar is missing. @ line 58, > column 17 > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException > > > > Are there something wrong in default pom.xml files? > > flume-ng-sinks/flume-hdfs-sink/pom.xml @ line 89 > <dependency> > <groupId>org.apache.hadoop</groupId> > <artifactId>${hadoop.common.artifact.id}</artifactId> > <optional>true</optional> > </dependency> > > flume-ng-sinks/flume-ng-hbase-sink/pom.xml @ line 96, > <dependency> > <groupId>org.apache.hadoop</groupId> > <artifactId>${hadoop.common.artifact.id}</artifactId> > <optional>true</optional> > </dependency> > > flume-ng-tests/pom.xml @ line 58 > <dependency> > <groupId>org.apache.hadoop</groupId> > <artifactId>${hadoop.common.artifact.id}</artifactId> > <optional>true</optional> > </dependency> > > > Many thanks! > Arthur > > > > > > > > > Also tried: > mvn clean install -Dhadoop.profile=hbase-98 -Dhbase.version=0.98.3-hadoop2 > [INFO] Scanning for projects... > [ERROR] The build could not read 3 projects -> [Help 1] > [ERROR] > [ERROR] The project org.apache.flume.flume-ng-sinks:flume-hdfs-sink:1.5.0 > (flume-ng-sinks/flume-hdfs-sink/pom.xml) has 2 errors > [ERROR] 'dependencies.dependency.artifactId' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar with value > '${hadoop.common.artifact.id}' does not match a valid id pattern. @ line 89, > column 19 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar is missing. @ line 87, > column 17 > [ERROR] > [ERROR] The project > org.apache.flume.flume-ng-sinks:flume-ng-hbase-sink:1.5.0 > (flume-ng-sinks/flume-ng-hbase-sink/pom.xml) has 2 errors > [ERROR] 'dependencies.dependency.artifactId' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar with value > '${hadoop.common.artifact.id}' does not match a valid id pattern. @ line 96, > column 19 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar is missing. @ line 94, > column 17 > [ERROR] > [ERROR] The project org.apache.flume:flume-ng-tests:1.5.0 > (flume-ng-tests/pom.xml) has 2 errors > [ERROR] 'dependencies.dependency.artifactId' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar with value > '${hadoop.common.artifact.id}' does not match a valid id pattern. @ line 60, > column 19 > [ERROR] 'dependencies.dependency.version' for > org.apache.hadoop:${hadoop.common.artifact.id}:jar is missing. @ line 58, > column 17 > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, please > read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException > > > > > >