Re: [VOTE] Release Apache Hadoop 3.2.2 - RC2

[Xiaoqiao He]

Thank Wei-Chiu, Chao and Stephen,

As Wei-Chiu suggests, HDFS-15680 is a blocker issue which breaks unit tests
of Azure. I will prepare the next RC3 once HDFS-15680 is resolved.

Regards,
- He Xiaoqiao

On Sat, Nov 14, 2020 at 7:57 AM Wei-Chiu Chuang 
wrote:

> I suggest we wait for HDFS-15680
> 
> (If I understand it correctly, it's a blocker)
>
> I wish to update jetty too in this release, but it's turning out to be
> more involved than i had anticipated.
>
>
> On Fri, Nov 13, 2020 at 3:22 PM Chao Sun  wrote:
>
>> +1 (non-binding) from me as well:
>>
>> - download the source and build it on mac successfully
>> - verified all md5, sha512 as well as signatures, however @Xiaoqiao I was
>> not able to find your public key in
>> http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS so had to use
>> the other link.
>> - started a single-node HDFS cluster with docker and verified that basic
>> operations are working
>> - checked the NN webui and it is working as expected
>>
>> Thanks,
>> Chao
>>
>>
>> On Tue, Nov 10, 2020 at 1:13 PM Stephen O'Donnell
>>  wrote:
>>
>>> I compiled a native build from source and started a non-HA, non-HA secure
>>> (TLS and Kerberos) and a HA hdfs cluster using docker.
>>>
>>> On each, I created a few files, looked around the webUI etc. Everything
>>> looks fine. The only strange thing, was on the Namenode webUI under the
>>> overview section - the "unknown" is a little strange, but I am not sure
>>> what is usually in there:
>>>
>>> Version: 3.2.2, rUnknown
>>> Compiled: Mon Nov 09 17:30:00 + 2020 by root from Unknown
>>>
>>> +1 from me.
>>>
>>> Stephen.
>>>
>>> On Sat, Nov 7, 2020 at 2:52 PM Xiaoqiao He 
>>> wrote:
>>>
>>> > Hi folks,
>>> >
>>> > The release candidate (RC2) for Hadoop-3.2.2 is available now.
>>> > There are two commits[1] differences between RC2 and RC1[2](Thanks
>>> Akira
>>> > Ajisaka for the report.):
>>> > * revert HADOOP-17306.
>>> > * include HDFS-15643.
>>> >
>>> > The RC2 is available at:
>>> > http://people.apache.org/~hexiaoqiao/hadoop-3.2.2-RC2
>>> > The RC2 tag in github is here:
>>> > https://github.com/apache/hadoop/tree/release-3.2.2-RC2
>>> > The maven artifacts are staged at:
>>> >
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1288
>>> >
>>> > You can find my public key at:
>>> > http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS or
>>> > https://people.apache.org/keys/committer/hexiaoqiao.asc directly.
>>> >
>>> > Please try the release and vote. The vote will close until 2020/11/14
>>> at
>>> > 00:00 CST.
>>> >
>>> > Thanks,
>>> > He Xiaoqiao
>>> >
>>> > [1]
>>> >
>>> >
>>> https://github.com/apache/hadoop/compare/release-3.2.2-RC1...release-3.2.2-RC2
>>> > [2]
>>> >
>>> >
>>> https://lists.apache.org/thread.html/rc7247434f5a77b6d0d1d1f3fcd6b6668eb431a5697e582a6338f0eb7%40%3Chdfs-dev.hadoop.apache.org%3E
>>> > [3]
>>> >
>>> https://issues.apache.org/jira/secure/Dashboard.jspa?selectPageId=12335948
>>> >
>>>
>>

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

[Apache Jenkins Server]

For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/

No changes




-1 overall


The following subsystems voted -1:
asflicense hadolint jshint pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml 
   hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml 
   
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml
 
   hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml 
   hadoop-tools/hadoop-azure/src/config/checkstyle.xml 
   hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml 
   hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml
 

Failed junit tests :

   hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints 
   
hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain 
   hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys 
   hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints 
   hadoop.hdfs.server.federation.router.TestRouterQuota 
   hadoop.hdfs.server.federation.router.TestRouterNamenodeHeartbeat 
   hadoop.hdfs.server.federation.resolver.order.TestLocalResolver 
   hadoop.hdfs.server.federation.resolver.TestMultipleDestinationResolver 
   hadoop.yarn.server.resourcemanager.TestClientRMService 
   hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer 
   hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter 
   hadoop.yarn.sls.appmaster.TestAMSimulator 
   hadoop.resourceestimator.solver.impl.TestLpSolver 
   hadoop.resourceestimator.service.TestResourceEstimatorService 
  

   jshint:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-patch-jshint.txt
  [208K]

   cc:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-compile-javac-root.txt
  [436K]

   checkstyle:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-checkstyle-root.txt
  [16M]

   hadolint:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-patch-hadolint.txt
  [4.0K]

   pathlen:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-patch-pylint.txt
  [60K]

   shellcheck:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-patch-shellcheck.txt
  [56K]

   shelldocs:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-patch-shelldocs.txt
  [8.0K]

   whitespace:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/whitespace-eol.txt
  [12M]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/whitespace-tabs.txt
  [1.3M]

   xml:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/xml.txt
  [4.0K]

   javadoc:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/diff-javadoc-javadoc-root.txt
  [20K]

   unit:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [272K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt
  [12K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
  [36K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [116K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/116/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase-tests.txt
  [20K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-lin

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

[Apache Jenkins Server]

For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/

[Nov 13, 2020 2:02:26 AM] (Takanobu Asanuma) HDFS-15538. Fix the documentation 
for dfs.namenode.replication.max-streams in hdfs-default.xml. Contributed by 
Xieming Li.
[Nov 13, 2020 8:22:35 PM] (noreply) HADOOP-17362. reduce RPC calls doing ls on 
HAR file (#2444). Contributed by Daryn Sharp and Ahmed Hussein




-1 overall


The following subsystems voted -1:
findbugs pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
 

Failed junit tests :

   hadoop.hdfs.server.blockmanagement.TestUnderReplicatedBlocks 
   hadoop.hdfs.server.federation.router.TestRouterRpc 
   hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor 
   hadoop.yarn.server.router.webapp.TestRouterWebServicesREST 
   hadoop.yarn.applications.distributedshell.TestDistributedShell 
   hadoop.tools.dynamometer.TestDynamometerInfra 
   hadoop.tools.dynamometer.TestDynamometerInfra 
   hadoop.fs.azure.TestNativeAzureFileSystemOperationsMocked 
   hadoop.fs.azure.TestNativeAzureFileSystemMocked 
   hadoop.fs.azure.TestBlobMetadata 
   hadoop.fs.azure.TestNativeAzureFileSystemConcurrency 
   hadoop.fs.azure.TestNativeAzureFileSystemFileNameCheck 
   hadoop.fs.azure.TestNativeAzureFileSystemContractMocked 
   hadoop.fs.azure.TestWasbFsck 
   hadoop.fs.azure.TestOutOfBandAzureBlobOperations 
   hadoop.yarn.sls.TestSLSRunner 
   hadoop.yarn.sls.appmaster.TestAMSimulator 
  

   cc:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/diff-compile-cc-root.txt
  [48K]

   javac:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/diff-compile-javac-root.txt
  [568K]

   checkstyle:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/diff-checkstyle-root.txt
  [16M]

   pathlen:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/diff-patch-pylint.txt
  [60K]

   shellcheck:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/diff-patch-shelldocs.txt
  [44K]

   whitespace:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/whitespace-eol.txt
  [13M]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/whitespace-tabs.txt
  [2.0M]

   xml:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/xml.txt
  [24K]

   javadoc:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/diff-javadoc-javadoc-root.txt
  [2.0M]

   findbugs:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/branch-findbugs-hadoop-common-project_hadoop-minikdc.txt
  [4.0K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/branch-findbugs-hadoop-common-project_hadoop-common.txt
  [88K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/branch-findbugs-hadoop-common-project_hadoop-nfs.txt
  [4.0K]

   unit:

   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [344K]
   
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/325/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs-rbf.txt
  [448K]