Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/ No changes -1 overall The following subsystems voted -1: asflicense hadolint mvnsite pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.fs.TestFileUtil hadoop.io.compress.snappy.TestSnappyCompressorDecompressor hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.TestRollingUpgrade hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat hadoop.hdfs.server.federation.router.TestRouterQuota hadoop.hdfs.server.federation.resolver.order.TestLocalResolver hadoop.yarn.server.resourcemanager.TestClientRMService hadoop.yarn.server.resourcemanager.monitor.invariants.TestMetricsInvariantChecker hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter hadoop.mapreduce.lib.input.TestLineRecordReader hadoop.mapred.TestLineRecordReader hadoop.yarn.sls.TestSLSRunner hadoop.resourceestimator.service.TestResourceEstimatorService hadoop.resourceestimator.solver.impl.TestLpSolver cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/diff-compile-javac-root.txt [476K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/diff-checkstyle-root.txt [14M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/diff-patch-hadolint.txt [4.0K] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-mvnsite-root.txt [560K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/diff-patch-shellcheck.txt [72K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-javadoc-root.txt [40K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [224K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [428K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [12K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [36K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt [20K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt [124K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt [104K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-tools_hadoop-azure.txt [20K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt [28K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/590/artifact/out/patch-unit-hadoop-tools_hadoop-resourceestimator.txt [16K] asflicense: http
Re: Hadoop-3.2.3 Release Update
Hi Brahma, I would like to use 3.2.3 for next minor release of Bigtop. https://issues.apache.org/jira/browse/BIGTOP-3605 Let me know if there is any blocker I can help. Thanks, Masatake Iwasaki On 2022/01/12 4:41, Brahma Reddy Battula wrote: Sorry, I was looking for some CVE’s fixes.will make it progress on this… On Tue, 11 Jan 2022 at 5:58 PM, Wei-Chiu Chuang wrote: Is this still making progress? On Tue, Oct 5, 2021 at 8:45 PM Brahma Reddy Battula wrote: Hi Akira, Thanks for your email!! I am evaluating the CVE’s which needs to go for this release.. Will update soon!! On Tue, 5 Oct 2021 at 1:46 PM, Akira Ajisaka wrote: Hi Brahma, What is the release process going on? Is there any blocker for the RC? -Akira On Wed, Sep 22, 2021 at 7:37 PM Xiaoqiao He wrote: Hi Brahma, The feature 'BPServiceActor processes commands from NameNode asynchronously' has been ready for both branch-3.2 and branch-3.2.3. While cherry-picking there is only minor conflict, So I checked in directly. BTW, run some unit tests and build pseudo cluster to verify, it seems to work fine. FYI. Regards, - He Xiaoqiao On Thu, Sep 16, 2021 at 10:52 PM Brahma Reddy Battula < bra...@apache.org wrote: Please go ahead. Let me know any help required on review. On Tue, Sep 14, 2021 at 6:57 PM Xiaoqiao He wrote: Hi Brahma, I plan to involve HDFS-14997 and related JIRAs if possible. I have resolved the conflict and verified them locally. It will include: HDFS-14997 HDFS-15075 HDFS-15651 HDFS-15113. I would like to hear some more response that if we have enough time to wait for it to be ready. Thanks. Best Regards, - He Xiaoqiao On Tue, Sep 14, 2021 at 3:39 PM Xiaoqiao He wrote: Hi Brahma, HDFS-15160 has checked in branch-3.2 & branch-3.2.3. FYI. On Tue, Sep 14, 2021 at 3:52 AM Brahma Reddy Battula < bra...@apache.org> wrote: Hi All, Waiting for the following jira to commit to hadoop-3.2.3 , mostly this can be done by this week,then I will try to create the RC next if there is no objection. https://issues.apache.org/jira/browse/HDFS-15160 On Mon, Aug 16, 2021 at 2:22 PM Brahma Reddy Battula < bra...@apache.org> wrote: @Akira Ajisaka and @Masatake Iwasaki Looks all are build related issues when you try with bigtop. We can discuss and prioritize this.. Will connect with you guys. On Mon, Aug 16, 2021 at 1:43 PM Masatake Iwasaki < iwasak...@oss.nttdata.co.jp> wrote: - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff This is for building hadoop-3.2.2 against zookeeper-3.4.14. we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13, while it would be harmless to add the exclusion even for zooekeeper-3.4.13. I filed HADOOP-17849 for this. On 2021/08/16 12:02, Masatake Iwasaki wrote: Thanks for bringing this up, Akira. Let me explain some background. - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff This is for building hadoop-3.2.2 against zookeeper-3.4.14. we do not see the issue usually since branch-3.2 uses zooekeper-3.4.13, while it would be harmless to add the exclusion even for zooekeeper-3.4.13. - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-fix-broken-dir-detection.diff - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch5-fix-kms-shellprofile.diff - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch6-fix-httpfs-sh.diff These are relevant to directory structure used by Bigtop package. If the fix does not break the tarball dist, it would be nice to have these on Hadoop too. - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch7-remove-phantomjs-in-yarn-ui.diff This is for aarch64 and ppe64le lacking required phantomjs. It is only acceptable for Bigtop not running tests of YARN-UI2 on packaging. Hadoop needs the phantomjs for testing YARN-UI2. On 2021/08/16 2:59, Brahma Reddy Battula wrote: Thanks, I'll check them out. On Sun, 15 Aug 2021 at 11:22 PM, Akira Ajisaka < aajis...@apache.org> wrote: Thanks Brahma for cutting branch-3.2.3. In Apache Bigtop, there are some patches applied to Hadoop 3.2.2. https://github.com/apache/bigtop/tree/master/bigtop-packages/src/common/hadoop In these patches, how about backporting the following issues to branch-3.2 and branch-3.2.3? - HADOOP-14922 - HADOOP-15939 - HADOOP-17569 In addition, there are some patches that don't have JIRA issue ID. Maybe we need to create JIRAs and fix those. - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch3-f
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/ [Mar 2, 2022 3:25:50 AM] (noreply) HDFS-16458. [SPS]: Fix bug for unit test of reconfiguring SPS mode (#4041) [Mar 2, 2022 10:16:24 AM] (gyorandr) YARN-10983. Follow-up changes for YARN-10904. Contributed by Benjamin Teke [Mar 2, 2022 10:25:56 AM] (noreply) HADOOP-18128. Fix typo issues of outputstream.md (#4025) [Mar 2, 2022 3:19:17 PM] (Szilard Nemeth) YARN-11042. Fix testQueueSubmitWithACLsEnabledWithQueueMapping in TestAppManager. Contributed by Tamas Domok -1 overall The following subsystems voted -1: blanks pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml Failed junit tests : hadoop.yarn.conf.TestYarnConfigurationFields cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/results-compile-cc-root.txt [96K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/results-compile-javac-root.txt [340K] blanks: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/blanks-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/blanks-tabs.txt [2.0M] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/results-checkstyle-root.txt [14M] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/results-pathlen.txt [16K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/results-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/results-shellcheck.txt [28K] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/results-javadoc-javadoc-root.txt [404K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/798/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt [52K] Powered by Apache Yetus 0.14.0-SNAPSHOT https://yetus.apache.org - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/267/ [Mar 1, 2022 11:48:10 AM] (noreply) HADOOP-18075. ABFS: Fix failure caused by listFiles() in ITestAbfsRestOperationException (#4040) [Mar 2, 2022 3:25:50 AM] (noreply) HDFS-16458. [SPS]: Fix bug for unit test of reconfiguring SPS mode (#4041) [Mar 2, 2022 10:16:24 AM] (gyorandr) YARN-10983. Follow-up changes for YARN-10904. Contributed by Benjamin Teke [Mar 2, 2022 10:25:56 AM] (noreply) HADOOP-18128. Fix typo issues of outputstream.md (#4025) [Mar 2, 2022 3:19:17 PM] (Szilard Nemeth) YARN-11042. Fix testQueueSubmitWithACLsEnabledWithQueueMapping in TestAppManager. Contributed by Tamas Domok [Mar 3, 2022 3:14:50 AM] (Sun Chao) Make upstream aware of 3.3.2 release [Error replacing 'FILE' - Workspace is not accessible] - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[ANNOUNCE] Apache Hadoop 3.3.2 release
Hi All, It gives me great pleasure to announce that the Apache Hadoop community has voted to release Apache Hadoop 3.3.2. This is the second stable release of Apache Hadoop 3.3 line. It contains 284 bug fixes, improvements and enhancements since 3.3.1. Users are encouraged to read the overview of major changes [1] since 3.3.1. For details of 284 bug fixes, improvements, and other enhancements since the previous 3.3.1 release, please check release notes [2] and changelog [3]. [1]: https://hadoop.apache.org/docs/r3.3.2/index.html [2]: http://hadoop.apache.org/docs/r3.3.2/hadoop-project-dist/hadoop-common/release/3.3.2/RELEASENOTES.3.3.2.html [3]: http://hadoop.apache.org/docs/r3.3.2/hadoop-project-dist/hadoop-common/release/3.3.2/CHANGELOG.3.3.2.html Many thanks to everyone who contributed to the release, and everyone in the Apache Hadoop community! This release is a direct result of your great contributions. Many thanks to everyone who helped in this release process! Many thanks to Viraj Jasani, Michael Stack, Masatake Iwasaki, Xiaoqiao He, Mukund Madhav Thakur, Wei-Chiu Chuang, Steve Loughran, Akira Ajisaka and other folks who helped for this release process. Best Regards, Chao
Re: [ANNOUNCE] Apache Hadoop 3.3.2 release
Thanks a lot for the tremendous work! On Fri, Mar 4, 2022 at 9:30 AM Chao Sun wrote: > Hi All, > > It gives me great pleasure to announce that the Apache Hadoop community has > voted to release Apache Hadoop 3.3.2. > > This is the second stable release of Apache Hadoop 3.3 line. It contains > 284 bug fixes, improvements and enhancements since 3.3.1. > > Users are encouraged to read the overview of major changes [1] since 3.3.1. > For details of 284 bug fixes, improvements, and other enhancements since > the previous 3.3.1 release, please check release notes [2] and changelog > [3]. > > [1]: https://hadoop.apache.org/docs/r3.3.2/index.html > [2]: > > http://hadoop.apache.org/docs/r3.3.2/hadoop-project-dist/hadoop-common/release/3.3.2/RELEASENOTES.3.3.2.html > [3]: > > http://hadoop.apache.org/docs/r3.3.2/hadoop-project-dist/hadoop-common/release/3.3.2/CHANGELOG.3.3.2.html > > Many thanks to everyone who contributed to the release, and everyone in the > Apache Hadoop community! This release is a direct result of your great > contributions. > > Many thanks to everyone who helped in this release process! > > Many thanks to Viraj Jasani, Michael Stack, Masatake Iwasaki, Xiaoqiao He, > Mukund Madhav Thakur, Wei-Chiu Chuang, Steve Loughran, Akira Ajisaka and > other folks who helped for this release process. > > Best Regards, > Chao >
[jira] [Created] (HDFS-16492) Add metrics and operation names for dataset lock
Yuanbo Liu created HDFS-16492: - Summary: Add metrics and operation names for dataset lock Key: HDFS-16492 URL: https://issues.apache.org/jira/browse/HDFS-16492 Project: Hadoop HDFS Issue Type: Sub-task Reporter: Yuanbo Liu -- This message was sent by Atlassian Jira (v8.20.1#820001) - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/ No changes -1 overall The following subsystems voted -1: hadolint mvnsite pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.io.compress.snappy.TestSnappyCompressorDecompressor hadoop.fs.TestFileUtil hadoop.hdfs.server.namenode.ha.TestBootstrapStandbyWithInProgressTailing hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.server.namenode.TestNameNodeRetryCacheMetrics hadoop.hdfs.server.namenode.TestDiskspaceQuotaUpdate hadoop.hdfs.server.namenode.ha.TestStandbyInProgressTail hadoop.hdfs.server.namenode.ha.TestDNFencing hadoop.hdfs.server.namenode.TestINodeAttributeProvider hadoop.hdfs.server.namenode.TestFsck hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain hadoop.hdfs.server.namenode.snapshot.TestRenameWithSnapshots hadoop.hdfs.server.namenode.TestParallelImageWrite hadoop.hdfs.server.namenode.ha.TestPipelinesFailover hadoop.hdfs.server.namenode.TestCacheDirectives hadoop.hdfs.server.namenode.TestProtectedDirectories hadoop.hdfs.server.namenode.ha.TestHASafeMode hadoop.hdfs.server.namenode.ha.TestConsistentReadsObserver hadoop.hdfs.server.namenode.ha.TestDFSUpgradeWithHA hadoop.hdfs.server.namenode.TestSnapshotPathINodes hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy hadoop.hdfs.server.namenode.ha.TestHAStateTransitions hadoop.hdfs.server.namenode.metrics.TestNameNodeMetrics hadoop.hdfs.server.namenode.TestLeaseManager hadoop.hdfs.server.namenode.TestFileTruncate hadoop.hdfs.server.namenode.TestGenericJournalConf cc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/diff-compile-javac-root.txt [476K] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/diff-checkstyle-root.txt [14M] hadolint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/diff-patch-hadolint.txt [4.0K] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-mvnsite-root.txt [564K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/pathlen.txt [12K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/diff-patch-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/diff-patch-shellcheck.txt [72K] whitespace: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/whitespace-eol.txt [12M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/whitespace-tabs.txt [1.3M] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-javadoc-root.txt [40K] unit: https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [224K] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt [1.1M] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-native-client.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-httpfs.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-nfs.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/591/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-api.txt [0] https://ci-ha
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/ [Mar 3, 2022 3:14:50 AM] (Sun Chao) Make upstream aware of 3.3.2 release [Mar 3, 2022 2:54:03 PM] (noreply) YARN-11076. Upgrade jQuery version in Yarn UI2. (#4046) [Mar 3, 2022 3:38:56 PM] (Szilard Nemeth) Revert "YARN-10947. Simplify AbstractCSQueue#initializeQueueState. Contributed by Andras Gyori" [Mar 3, 2022 3:44:12 PM] (Szilard Nemeth) YARN-10947. Simplify AbstractCSQueue#initializeQueueState. Contributed by Andras Gyori [Mar 3, 2022 6:44:28 PM] (noreply) HADOOP-18150. Fix ITestAuditManagerDisabled test in S3A. (#4044) -1 overall The following subsystems voted -1: blanks compile golang mvninstall mvnsite pathlen spotbugs unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) shadedclient Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml mvninstall: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/patch-mvninstall-root.txt [1.4M] compile: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/patch-compile-root.txt [256K] cc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/patch-compile-root.txt [256K] golang: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/patch-compile-root.txt [256K] javac: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/patch-compile-root.txt [256K] blanks: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/blanks-eol.txt [13M] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/blanks-tabs.txt [2.0M] checkstyle: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/results-checkstyle-root.txt [14M] mvnsite: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/patch-mvnsite-root.txt [568K] pathlen: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/results-pathlen.txt [16K] pylint: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/results-pylint.txt [20K] shellcheck: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/results-shellcheck.txt [28K] xml: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/xml.txt [24K] javadoc: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/patch-javadoc-root.txt [0] spotbugs: https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-build-tools.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-project.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-annotations.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-project-dist.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-assemblies.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-maven-plugins.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-minikdc.txt [0] https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/799/artifact/out/branch-spotbugs-hadoop-common-project_hadoop-auth.txt [0] https://ci-hadoop.ap
Re: [ANNOUNCE] Apache Hadoop 3.3.2 release
Great work! Thanks, Chao! Chao Sun 于2022年3月4日周五 09:30写道： > Hi All, > > It gives me great pleasure to announce that the Apache Hadoop community has > voted to release Apache Hadoop 3.3.2. > > This is the second stable release of Apache Hadoop 3.3 line. It contains > 284 bug fixes, improvements and enhancements since 3.3.1. > > Users are encouraged to read the overview of major changes [1] since 3.3.1. > For details of 284 bug fixes, improvements, and other enhancements since > the previous 3.3.1 release, please check release notes [2] and changelog > [3]. > > [1]: https://hadoop.apache.org/docs/r3.3.2/index.html > [2]: > > http://hadoop.apache.org/docs/r3.3.2/hadoop-project-dist/hadoop-common/release/3.3.2/RELEASENOTES.3.3.2.html > [3]: > > http://hadoop.apache.org/docs/r3.3.2/hadoop-project-dist/hadoop-common/release/3.3.2/CHANGELOG.3.3.2.html > > Many thanks to everyone who contributed to the release, and everyone in the > Apache Hadoop community! This release is a direct result of your great > contributions. > > Many thanks to everyone who helped in this release process! > > Many thanks to Viraj Jasani, Michael Stack, Masatake Iwasaki, Xiaoqiao He, > Mukund Madhav Thakur, Wei-Chiu Chuang, Steve Loughran, Akira Ajisaka and > other folks who helped for this release process. > > Best Regards, > Chao >
[jira] [Created] (HDFS-16493) [SBN Read]When fast path tail enabled, standby or observer namenode may read uncommitted data
liutongwei created HDFS-16493:
-
Summary: [SBN Read]When fast path tail enabled, standby or
observer namenode may read uncommitted data
Key: HDFS-16493
URL: https://issues.apache.org/jira/browse/HDFS-16493
Project: Hadoop HDFS
Issue Type: Bug
Components: journal-node, namanode
Reporter: liutongwei
Attachments: example.patch
Although fast path tail use quorum read to pull edit log, it seem like is can
read uncommitted data in some corner case.
Here is an example. Suppose we have three JN, their init state is:
{code:java}
epoch 1
JN1 [1-3](in-progress)
JN2 [1-3](in-progress)
JN3 [1-4](in-progress)
Note that, in epoch 1 txid 1-3 was committed, and txid 4 not.
{code}
When a failover occur, if a new writer cannot contact to JN3 for network
partition, and finish the recovery stage, and write a new txid 4 in epoch 2,
which value not equal to JN3's.
{code:java}
epcho 2
JN1 [1-3](finalized) [4-4](inprogress)
JN2 [1-3](finalized) [4-4](inprogress)
JN3 [1-4](inprogress)
Note that, in JN3 txid4's value not equal to other JN.
{code}
Now there is a read namenode to pull edits, and it contact to JN3 and JN2, it
got majority response. But it got logs of same length but different content.And
no more information to choose which log is right. If we choose JN3, we got meta
data corruption.
There is a test example patch [^example.patch] for running and debug.
For fix it i think we should add finalized state to
{{{}GetJournaledEditsResponseProto{}}}, so we can discard the fault log.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
