Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2018-01-29 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/119/ [Jan 28, 2018 4:31:21 AM] (stevel) HADOOP-15039/HADOOP-15189. Move SemaphoredDelegatingExecutor to -1 overall The following subsystems voted -1: asflicense unit xml The following subsystems vo

does it make sense to get remaining space by sum all the ones of the datanode

2018-01-29 Thread Xie Gang
Hello, We recently hit a issue that almost all the disk of the datanode got full even we configured the du .reserved. After tracking down the code, found that when we choose a target datanode and check if it's good candidate for block allocation (isGoodTarget()), it only checks if the total left

Re: does it make sense to get remaining space by sum all the ones of the datanode

2018-01-29 Thread Vinayakumar B
in which version of Hadoop you are seeing this? -Vinay On 29 Jan 2018 3:26 pm, "Xie Gang" wrote: Hello, We recently hit a issue that almost all the disk of the datanode got full even we configured the du .reserved. After tracking down the code, found that when we choose a target datanode and

Re: does it make sense to get remaining space by sum all the ones of the datanode

2018-01-29 Thread Xie Gang
2.4 and 2.6: public long getRemaining(StorageType t) { long remaining = 0; for(DatanodeStorageInfo s : getStorageInfos()) { if (s.getStorageType() == t) { remaining += s.getRemaining(); } } return remaining; } On Mon, Jan 29, 2018 at 6:12 PM, Vinayakumar B wrote: > in whi

Re: does it make sense to get remaining space by sum all the ones of the datanode

2018-01-29 Thread Vinayakumar B
in the latest code this is is not applicable. each storage is chosen and space of that storage is checked. On 29 Jan 2018 3:53 pm, "Xie Gang" wrote: > 2.4 and 2.6: > > public long getRemaining(StorageType t) { > long remaining = 0; > for(DatanodeStorageInfo s : getStorageInfos()) { > if

[jira] [Created] (HDFS-13077) [SPS]: Fix review comments of external storage policy satisfier

2018-01-29 Thread Rakesh R (JIRA)
Rakesh R created HDFS-13077: --- Summary: [SPS]: Fix review comments of external storage policy satisfier Key: HDFS-13077 URL: https://issues.apache.org/jira/browse/HDFS-13077 Project: Hadoop HDFS Is

[jira] [Created] (HDFS-13078) Ozone: Ratis read fail because stream is closed before the reply is received

2018-01-29 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created HDFS-13078: Summary: Ozone: Ratis read fail because stream is closed before the reply is received Key: HDFS-13078 URL: https://issues.apache.org/jira/browse/HDFS-13078 Pr

[jira] [Created] (HDFS-13079) Provide a config to start namenode in safemode state upto a certain transaction id

2018-01-29 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created HDFS-13079: Summary: Provide a config to start namenode in safemode state upto a certain transaction id Key: HDFS-13079 URL: https://issues.apache.org/jira/browse/HDFS-13079

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-01-29 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/672/ [Jan 29, 2018 6:12:05 AM] (xiao) HDFS-13065. TestErasureCodingMultipleRacks#testSkewedRack3 is failing. [Jan 29, 2018 6:19:49 AM] (xiao) HDFS-12974. Exception message is not printed when creating an encryp

Re: performance about writing data to HDFS

2018-01-29 Thread Miklos Szegedi
Hello, Here is an example. You can set an initial low replication like this code does: https://github.com/apache/hadoop/blob/56feaa40bb94fcaa96ae668eebfabec4611928c0/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-uploader/src/main/java/org/apache/hadoop/mapred/uploader/F

[jira] [Created] (HDFS-13080) Ozone: Make finalhash in ContainerInfo of StorageContainerDatanodeProtocol.proto optional

2018-01-29 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-13080: -- Summary: Ozone: Make finalhash in ContainerInfo of StorageContainerDatanodeProtocol.proto optional Key: HDFS-13080 URL: https://issues.apache.org/jira/browse/HDFS-13080 P

[jira] [Created] (HDFS-13081) Datanode#checkSecureConfig should check HTTPS and SASL encryption

2018-01-29 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-13081: - Summary: Datanode#checkSecureConfig should check HTTPS and SASL encryption Key: HDFS-13081 URL: https://issues.apache.org/jira/browse/HDFS-13081 Project: Hadoop HDFS

[jira] [Created] (HDFS-13082) cookieverf mismatch error over NFS gateway on Linux

2018-01-29 Thread Dan Moraru (JIRA)
Dan Moraru created HDFS-13082: - Summary: cookieverf mismatch error over NFS gateway on Linux Key: HDFS-13082 URL: https://issues.apache.org/jira/browse/HDFS-13082 Project: Hadoop HDFS Issue Type:

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

2018-01-29 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/120/ No changes [Error replacing 'FILE' - Workspace is not accessible] - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For