[jira] [Created] (HDFS-15240) dirty

2020-03-25 Thread HuangTao (Jira) 
HuangTao created HDFS-15240:
---

 Summary: dirty
 Key: HDFS-15240
 URL: https://issues.apache.org/jira/browse/HDFS-15240
 Project: Hadoop HDFS
  Issue Type: Bug
  Components: datanode, erasure-coding
Reporter: HuangTao






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (HDFS-15241) Distcp print wrong log info when use -log

2020-03-25 Thread liuyanyu (Jira) 
liuyanyu created HDFS-15241:
---

 Summary: Distcp print wrong log info when use -log
 Key: HDFS-15241
 URL: https://issues.apache.org/jira/browse/HDFS-15241
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: distcp
Affects Versions: 3.1.1
Reporter: liuyanyu
 Attachments: image-2020-03-25-17-28-33-394.png

when run distcp with -log /logpath -v, distcp will print copy status and file 
info to /logpath, but print log with wrong file zise. The logs print as follows:

FILE_COPIED: source=hdfs://ns1/test/stax2-api-3.1.4.jar, size=161867 --> 
target=hdfs://ns1/tmp/target/stax2-api-3.1.4.jar, size=0

As I analysis ,the root cause is as follows:

targrtFileStatus got before copying. So targrtFileStatus is null. Here should 
get targrtFileStatus again after file copying.

!image-2020-03-25-17-28-33-394.png!



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch2.10+JDK7 on Linux/x86

2020-03-25 Thread Apache Jenkins Server 
For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/

No changes




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml
 
   hadoop-tools/hadoop-azure/src/config/checkstyle-suppressions.xml 
   hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml
 

FindBugs :

   
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client
 
   Boxed value is unboxed and then immediately reboxed in 
org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result,
 byte[], byte[], KeyConverter, ValueConverter, boolean) At 
ColumnRWHelper.java:then immediately reboxed in 
org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result,
 byte[], byte[], KeyConverter, ValueConverter, boolean) At 
ColumnRWHelper.java:[line 335] 

Failed junit tests :

   hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys 
   hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints 
   hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints 
   hadoop.yarn.client.api.impl.TestTimelineClientV2Impl 
   hadoop.registry.secure.TestSecureLogins 
   hadoop.yarn.client.api.impl.TestAMRMProxy 
   hadoop.tools.TestDistCpSystem 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-compile-cc-root-jdk1.7.0_95.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-compile-javac-root-jdk1.7.0_95.txt
  [324K]

   cc:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-compile-cc-root-jdk1.8.0_242.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-compile-javac-root-jdk1.8.0_242.txt
  [304K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-checkstyle-root.txt
  [16M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-patch-hadolint.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-patch-pylint.txt
  [24K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-patch-shellcheck.txt
  [56K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-patch-shelldocs.txt
  [8.0K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/whitespace-eol.txt
  [12M]
   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/whitespace-tabs.txt
  [1.3M]

   xml:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/xml.txt
  [12K]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-client-warnings.html
  [8.0K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-javadoc-javadoc-root-jdk1.7.0_95.txt
  [16K]
   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/diff-javadoc-javadoc-root-jdk1.8.0_242.txt
  [1.1M]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
  [232K]
   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs_src_contrib_bkjournal.txt
  [12K]
   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-common.txt
  [36K]
   
https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/635/artifact/out/patch-unit-hadoop-yarn-projec

[jira] [Created] (HDFS-15242) Add metrics for operations hold lock times of FsDatasetImpl

2020-03-25 Thread Xiaoqiao He (Jira) 
Xiaoqiao He created HDFS-15242:
--

 Summary: Add metrics for operations hold lock times of 
FsDatasetImpl
 Key: HDFS-15242
 URL: https://issues.apache.org/jira/browse/HDFS-15242
 Project: Hadoop HDFS
  Issue Type: Improvement
  Components: datanode
Reporter: Xiaoqiao He
Assignee: Xiaoqiao He


Some operations of FsDatasetImpl need to hold Lock, and sometimes it costs long 
time to execute since it include IO operation in Lock. I propose to add metrics 
for this operations then it could be more convenient for monitor and dig 
bottleneck.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/aarch

2020-03-25 Thread Apache Jenkins Server 
ARM Build Details : 
https://builds.apache.org/job/Hadoop-qbt-linux-ARM-trunk/149/

 
CHANGES 
 

[Mar 24, 2020 5:26:09 PM] (ayushsaxena) HDFS-13377. The owner of folder can set 
quota for his sub folder.
[Mar 24, 2020 5:47:22 PM] (github) HDFS-15219. DFS Client will stuck when 
ResponseProcessor.run throw Error
[Mar 24, 2020 9:50:15 PM] (github) HDFS-15215. The Timestamp for longest 
write/read lock held log is wrong
[Mar 25, 2020 10:39:35 AM] (github) HADOOP-16930. Add hadoop-aws documentation 
for
[Mar 25, 2020 11:20:19 AM] (sunilg) YARN-9879. Allow multiple leaf queues with 
the same name in
[Mar 25, 2020 3:06:58 PM] (github) HADOOP-16938. Make non-HA proxy providers 
pluggable


 
TESTS 
 

There are  total tests of which  test(s) failed and  test(s) were skipped.  

 
FAILED TESTS 
 

No tests ran.

Complete test report can be accessed at 

IN ATTACHMENT COMPRESSED BUILD LOGS.<>

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org