Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/ No changes -1 overall The following subsystems voted -1: asflicense hadolint mvnsite pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.ipc.TestIPC hadoop.fs.TestFileUtil hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.server.namenode.snapshot.TestSnapshotDeletion hadoop.hdfs.TestDFSInotifyEventInputStream hadoop.hdfs.server.namenode.TestNameNodeHttpServerXFrame hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.balancer.TestBalancerWithHANameNodes hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.mapreduce.lib.input.TestLineRecordReader hadoop.mapred.TestLineRecordReader hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.tools.TestDistCpSystem hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.resourceestimator.solver.impl.TestLpSolver hadoop.yarn.sls.TestSLSRunner hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestNumaResourceAllocator hadoop.yarn.server.nodemanager.containermanager.linux.resources.TestNumaResourceHandlerImpl hadoop.yarn.server.resourcemanager.TestClientRMService hadoop.yarn.server.resourcemanager.recovery.TestFSRMStateStore hadoop.yarn.server.resourcemanager.monitor.invariants.TestMetricsInvariantChecker cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/diff-compile-javac-root.txt [488K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/diff-checkstyle-root.txt [14M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/diff-patch-hadolint.txt [4.0K] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-mvnsite-root.txt [572K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/diff-patch-shellcheck.txt [72K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-javadoc-root.txt [36K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [224K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [460K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [36K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [16K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt [104K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt [20K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/1492/artifact/out/patch-unit-hadoop-tools_hadoop-distcp.txt [32K]
Re: [VOTE] Release Apache Hadoop Thirdparty 1.3.0
looks like there is some inability of maven to handle the double invocation where the manifest LICENSE.txt is built by copying in /LICENSE-binary I've just checked and this happens on branch 1.2.0 as well (64b6721). So it is not a regression, just a way for the build process to fail. because its not a regression, I don't think its a blocker. but we should mention it in our release notes about how to build this - arm64 doesn't work - you can't do package and install at the same tiime. Doesn't "mvn package" do an install first? I get confused here On Thu, 22 Aug 2024 at 02:36, slfan1989 wrote: > I encountered some issues while trying to compile with version > hadoop-thirdparty-1.3.0-RC1. > > Using mvn package clean -DskipTests succeeds in compiling, but using mvn > install package -DskipTests fails to compile. > > mvn clean package -DskipTests > > [INFO] > > [INFO] Reactor Summary for Apache Hadoop Third-party Libs 1.3.0: > [INFO] > [INFO] Apache Hadoop Third-party Libs . SUCCESS [ > 0.862 s] > [INFO] Apache Hadoop shaded Protobuf .. SUCCESS [ > 1.521 s] > [INFO] Apache Hadoop shaded Guava . SUCCESS [ > 1.027 s] > [INFO] Apache Hadoop shaded Avro 1.11 . SUCCESS [ > 0.408 s] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 4.261 s > [INFO] Finished at: 2024-08-22T09:32:46+08:00 > [INFO] > > > mvn install package -DskipTests > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-shade-plugin:3.1.1:shade (shade-protobuf) on > project hadoop-shaded-protobuf_3_25: Error creating shaded jar: duplicate > entry: META-INF/LICENSE.txt -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the -e > switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, > please read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with the > command > [ERROR] mvn -rf :hadoop-shaded-protobuf_3_25 > > > Shilun Fan. > > On Thu, Aug 22, 2024 at 9:07 AM Mukund Madhav Thakur > > +1 ( binding) > > > > On Tue, Aug 20, 2024 at 9:16 AM Steve Loughran > > wrote: > > > > > Mukund and I have built a release candidate (RC1) for Hadoop-Thirdparty > > > 1.3.0. > > > > > > The RC is available at: > > > > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.3.0-RC1/ > > > > > > The git tag is release-1.3.0-RC1, commit > > > 0fd62903b071b5186f31b7030ce42e1c00f6bb6a > > > > > > The maven artifacts are staged at > > > > https://repository.apache.org/content/repositories/orgapachehadoop-1420/ > > > > > > We've already been using the snapshot artifacts on hadoop trunk for the > > > protobuf update for a while, so we know that bit seems good. > > > Once this release is out we can adopt it in the 3.4.1 release (for > better > > > java 8 compatibility as well as updates). > > > > > > I have created a hadoop PR for the switch > > > https://github.com/apache/hadoop/pull/7007 > > > > > > This also updates the (transitive) binary license. The latest patch > also > > > tries to update docker to the same version of protoc -which hasn't been > > > done until now. Let's see what happens. > > > > > > Please try the release and vote. The vote will run for 5 days. > > > > > > Here is my vote: > > > > > > +1 (binding) > > > > > > > >
Re: [VOTE] Release Apache Hadoop Thirdparty 1.3.0
My oversight led to executing the wrong command. The command I actually needed to run was `mvn clean install -DskipTests`, which is the expected one. I will continue with my verification. [INFO] [INFO] Reactor Summary for Apache Hadoop Third-party Libs 1.3.0: [INFO] [INFO] Apache Hadoop Third-party Libs . SUCCESS [ 0.958 s] [INFO] Apache Hadoop shaded Protobuf .. SUCCESS [ 1.481 s] [INFO] Apache Hadoop shaded Guava . SUCCESS [ 1.043 s] [INFO] Apache Hadoop shaded Avro 1.11 . SUCCESS [ 0.350 s] [INFO] [INFO] BUILD SUCCESS [INFO] [INFO] Total time: 4.385 s [INFO] Finished at: 2024-08-22T22:54:38+08:00 [INFO] On Thu, Aug 22, 2024 at 9:35 AM slfan1989 wrote: > > I encountered some issues while trying to compile with version > hadoop-thirdparty-1.3.0-RC1. > > Using mvn package clean -DskipTests succeeds in compiling, but using mvn > install package -DskipTests fails to compile. > > mvn clean package -DskipTests > > [INFO] > > [INFO] Reactor Summary for Apache Hadoop Third-party Libs 1.3.0: > [INFO] > [INFO] Apache Hadoop Third-party Libs . SUCCESS [ > 0.862 s] > [INFO] Apache Hadoop shaded Protobuf .. SUCCESS [ > 1.521 s] > [INFO] Apache Hadoop shaded Guava . SUCCESS [ > 1.027 s] > [INFO] Apache Hadoop shaded Avro 1.11 . SUCCESS [ > 0.408 s] > [INFO] > > [INFO] BUILD SUCCESS > [INFO] > > [INFO] Total time: 4.261 s > [INFO] Finished at: 2024-08-22T09:32:46+08:00 > [INFO] > > > mvn install package -DskipTests > > [ERROR] Failed to execute goal > org.apache.maven.plugins:maven-shade-plugin:3.1.1:shade (shade-protobuf) on > project hadoop-shaded-protobuf_3_25: Error creating shaded jar: duplicate > entry: META-INF/LICENSE.txt -> [Help 1] > [ERROR] > [ERROR] To see the full stack trace of the errors, re-run Maven with the > -e switch. > [ERROR] Re-run Maven using the -X switch to enable full debug logging. > [ERROR] > [ERROR] For more information about the errors and possible solutions, > please read the following articles: > [ERROR] [Help 1] > http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException > [ERROR] > [ERROR] After correcting the problems, you can resume the build with the > command > [ERROR] mvn -rf :hadoop-shaded-protobuf_3_25 > > > Shilun Fan. > > On Thu, Aug 22, 2024 at 9:07 AM Mukund Madhav Thakur > > +1 ( binding) >> >> On Tue, Aug 20, 2024 at 9:16 AM Steve Loughran >> wrote: >> >> > Mukund and I have built a release candidate (RC1) for Hadoop-Thirdparty >> > 1.3.0. >> > >> > The RC is available at: >> > >> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.3.0-RC1/ >> > >> > The git tag is release-1.3.0-RC1, commit >> > 0fd62903b071b5186f31b7030ce42e1c00f6bb6a >> > >> > The maven artifacts are staged at >> > >> https://repository.apache.org/content/repositories/orgapachehadoop-1420/ >> > >> > We've already been using the snapshot artifacts on hadoop trunk for the >> > protobuf update for a while, so we know that bit seems good. >> > Once this release is out we can adopt it in the 3.4.1 release (for >> better >> > java 8 compatibility as well as updates). >> > >> > I have created a hadoop PR for the switch >> > https://github.com/apache/hadoop/pull/7007 >> > >> > This also updates the (transitive) binary license. The latest patch also >> > tries to update docker to the same version of protoc -which hasn't been >> > done until now. Let's see what happens. >> > >> > Please try the release and vote. The vote will run for 5 days. >> > >> > Here is my vote: >> > >> > +1 (binding) >> > >> >>
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1677/ [Aug 21, 2024, 1:15:28 PM] (github) HADOOP-18542. Keep MSI tenant ID and client ID optional (#4262) [Aug 21, 2024, 8:11:32 PM] (github) HDFS-17546. Follow-up backport from branch3.3 (#6908) -1 overall The following subsystems voted -1: blanks hadolint pathlen spotbugs unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) shadedclient unit Specific tests: XML : Parsing Error(s): hadoop-common-project/hadoop-common/src/test/resources/xml/external-dtd.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml spotbugs : module:hadoop-hdfs-project/hadoop-hdfs-httpfs Redundant nullcheck of xAttrs, which is known to be non-null in org.apache.hadoop.fs.http.client.HttpFSFileSystem.getXAttr(Path, String) Redundant null check at HttpFSFileSystem.java:is known to be non-null in org.apache.hadoop.fs.http.client.HttpFSFileSystem.getXAttr(Path, String) Redundant null check at HttpFSFileSystem.java:[line 1374] spotbugs : module:hadoop-yarn-project/hadoop-yarn org.apache.hadoop.yarn.service.ServiceScheduler$1.load(ConfigFile) may return null, but is declared @Nonnull At ServiceScheduler.java:is declared @Nonnull At ServiceScheduler.java:[line 555] spotbugs : module:hadoop-hdfs-project Redundant nullcheck of xAttrs, which is known to be non-null in org.apache.hadoop.fs.http.client.HttpFSFileSystem.getXAttr(Path, String) Redundant null check at HttpFSFileSystem.java:is known to be non-null in org.apache.hadoop.fs.http.client.HttpFSFileSystem.getXAttr(Path, String) Redundant null check at HttpFSFileSystem.java:[line 1374] spotbugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications org.apache.hadoop.yarn.service.ServiceScheduler$1.load(ConfigFile) may return null, but is declared @Nonnull At ServiceScheduler.java:is declared @Nonnull At ServiceScheduler.java:[line 555] spotbugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services org.apache.hadoop.yarn.service.ServiceScheduler$1.load(ConfigFile) may return null, but is declared @Nonnull At ServiceScheduler.java:is declared @Nonnull At ServiceScheduler.java:[line 555] spotbugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-services/hadoop-yarn-services-core org.apache.hadoop.yarn.service.ServiceScheduler$1.load(ConfigFile) may return null, but is declared @Nonnull At ServiceScheduler.java:is declared @Nonnull At ServiceScheduler.java:[line 555] spotbugs : module:hadoop-yarn-project org.apache.hadoop.yarn.service.ServiceScheduler$1.load(ConfigFile) may return null, but is declared @Nonnull At ServiceScheduler.java:is declared @Nonnull At ServiceScheduler.java:[line 555] spotbugs : module:root Redundant nullcheck of xAttrs, which is known to be non-null in org.apache.hadoop.fs.http.client.HttpFSFileSystem.getXAttr(Path, String) Redundant null check at HttpFSFileSystem.java:is known to be non-null in org.apache.hadoop.fs.http.client.HttpFSFileSystem.getXAttr(Path, String) Redundant null check at HttpFSFileSystem.java:[line 1374] org.apache.hadoop.yarn.service.ServiceScheduler$1.load(ConfigFile) may return null, but is declared @Nonnull At ServiceScheduler.java:is declared @Nonnull At ServiceScheduler.java:[line 555] Failed junit tests : hadoop.hdfs.TestRollingUpgrade hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1677/artifact/out/results-compile-cc-root.txt [96K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/1677/artifact/out/results-compile-javac-root.txt [12K] blanks: h
Re: [VOTE] Release Apache Hadoop Thirdparty 1.3.0
+1 (binding). Thanks Steve and Mukund for driving this release. Best Regards - He Xiaoqiao On Wed, Aug 21, 2024 at 11:51 PM Mukund Madhav Thakur wrote: > +1 ( binding) > > On Tue, Aug 20, 2024 at 9:16 AM Steve Loughran > > wrote: > > > Mukund and I have built a release candidate (RC1) for Hadoop-Thirdparty > > 1.3.0. > > > > The RC is available at: > > > https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.3.0-RC1/ > > > > The git tag is release-1.3.0-RC1, commit > > 0fd62903b071b5186f31b7030ce42e1c00f6bb6a > > > > The maven artifacts are staged at > > https://repository.apache.org/content/repositories/orgapachehadoop-1420/ > > > > We've already been using the snapshot artifacts on hadoop trunk for the > > protobuf update for a while, so we know that bit seems good. > > Once this release is out we can adopt it in the 3.4.1 release (for better > > java 8 compatibility as well as updates). > > > > I have created a hadoop PR for the switch > > https://github.com/apache/hadoop/pull/7007 > > > > This also updates the (transitive) binary license. The latest patch also > > tries to update docker to the same version of protoc -which hasn't been > > done until now. Let's see what happens. > > > > Please try the release and vote. The vote will run for 5 days. > > > > Here is my vote: > > > > +1 (binding) > > >
