Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Mingliang Liu
+1 (binding) Thanks! On Tue, Mar 20, 2018 at 11:20 AM, Owen O'Malley wrote: > All, > > Following our discussions on the previous thread (Merging branch HDFS-7240 > to trunk), I'd like to propose the following: > > * HDSL become a subproject of Hadoop. > * HDSL will release separately from Hadoo

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Vinayakumar B
+1 for the subproject of HDSL Instead of maintaining directly as part of main repo itself, may be we can explore 'git submodules' -Vinay On 23 Mar 2018 11:47 am, "Takanobu Asanuma" wrote: +1 (non-binding). Thanks, -Takanobu Asanuma > All, > > Following our discussions on the previous threa

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Takanobu Asanuma
+1 (non-binding). Thanks, -Takanobu Asanuma > All, > > Following our discussions on the previous thread (Merging branch HDFS-7240 > to trunk), I'd like to propose the following: > > * HDSL become a subproject of Hadoop. > * HDSL will release separately from Hadoop. Hadoop releases will not > c

Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64

2018-03-22 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-trunk-win/414/ [Mar 21, 2018 8:53:35 PM] (xyao) HDFS-11043. TestWebHdfsTimeouts fails. Contributed by Xiaoyu Yao and [Mar 21, 2018 10:19:20 PM] (jlowe) YARN-8054. Improve robustness of the LocalDirsHandlerService [Mar 21, 2018 11:46:03 P

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Nandakumar Vadivelu
+1 (non-binding) On 3/23/18, 8:23 AM, "dujunp...@gmail.com on behalf of 俊平堵" wrote: I think the proposal here is the right way to get consensus from each part of community. +1 (binding) Thanks Owen for driving this. Thanks, Junping 2018-03-21 2:20 GM

[jira] [Created] (HDFS-13338) Native Build failed in hadoop-hdfs-native-client

2018-03-22 Thread Takanobu Asanuma (JIRA)
Takanobu Asanuma created HDFS-13338: --- Summary: Native Build failed in hadoop-hdfs-native-client Key: HDFS-13338 URL: https://issues.apache.org/jira/browse/HDFS-13338 Project: Hadoop HDFS Is

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread 俊平堵
I think the proposal here is the right way to get consensus from each part of community. +1 (binding) Thanks Owen for driving this. Thanks, Junping 2018-03-21 2:20 GMT+08:00 Owen O'Malley : > All, > > Following our discussions on the previous thread (Merging branch HDFS-7240 > to trunk), I'd l

[jira] [Created] (HDFS-13337) Backport HDFS-4275 to branch-2.9

2018-03-22 Thread JIRA
Íñigo Goiri created HDFS-13337: -- Summary: Backport HDFS-4275 to branch-2.9 Key: HDFS-13337 URL: https://issues.apache.org/jira/browse/HDFS-13337 Project: Hadoop HDFS Issue Type: Test

[jira] [Created] (HDFS-13336) Test cases of TestWriteToReplica failed in windows

2018-03-22 Thread Xiao Liang (JIRA)
Xiao Liang created HDFS-13336: - Summary: Test cases of TestWriteToReplica failed in windows Key: HDFS-13336 URL: https://issues.apache.org/jira/browse/HDFS-13336 Project: Hadoop HDFS Issue Type:

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Chen Liang
+1 (non-binding). Thanks, Chen > On Mar 20, 2018, at 11:20 AM, Owen O'Malley wrote: > > All, > > Following our discussions on the previous thread (Merging branch HDFS-7240 > to trunk), I'd like to propose the following: > > * HDSL become a subproject of Hadoop. > * HDSL will release separate

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Suresh Srinivas
+1 (Binding) Regards, Suresh On Tue, Mar 20, 2018 at 11:20 AM, Owen O'Malley wrote: > All, > > Following our discussions on the previous thread (Merging branch HDFS-7240 > to trunk), I'd like to propose the following: > > * HDSL become a subproject of Hadoop. > * HDSL will release separately fr

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Hanisha Koneru
+1 (non-binding). Thanks, Hanisha On 3/22/18, 4:06 PM, "Elek, Marton" wrote: > > > We need to do this since the source tarball is our official Apache >release > > artifact, the rest are convenience binaries. So the Maven profile is > > insufficient for this. > >It's a very good point, but a

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Elek, Marton
> We need to do this since the source tarball is our official Apache release > artifact, the rest are convenience binaries. So the Maven profile is > insufficient for this. It's a very good point, but actually it could be handled with maven profile and assembly descriptors. I created HDFS-133

[jira] [Created] (HDFS-13335) Ozone: remove hdsl/cblock/ozone source code from the official apache source release artifact

2018-03-22 Thread Elek, Marton (JIRA)
Elek, Marton created HDFS-13335: --- Summary: Ozone: remove hdsl/cblock/ozone source code from the official apache source release artifact Key: HDFS-13335 URL: https://issues.apache.org/jira/browse/HDFS-13335

[jira] [Created] (HDFS-13334) Ozone: Store only datanode UUID in datanode.id file

2018-03-22 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-13334: -- Summary: Ozone: Store only datanode UUID in datanode.id file Key: HDFS-13334 URL: https://issues.apache.org/jira/browse/HDFS-13334 Project: Hadoop HDFS Issue Typ

[jira] [Created] (HDFS-13333) Ozone: Introduce a new SCM Exception which will be thrown when mandatory property is missing

2018-03-22 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-1: -- Summary: Ozone: Introduce a new SCM Exception which will be thrown when mandatory property is missing Key: HDFS-1 URL: https://issues.apache.org/jira/browse/HDFS-1

[jira] [Created] (HDFS-13332) Ozone: update log4j.properties changes for hdsl/ozone.

2018-03-22 Thread Xiaoyu Yao (JIRA)
Xiaoyu Yao created HDFS-13332: - Summary: Ozone: update log4j.properties changes for hdsl/ozone. Key: HDFS-13332 URL: https://issues.apache.org/jira/browse/HDFS-13332 Project: Hadoop HDFS Issue Ty

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

2018-03-22 Thread Elek, Marton
+1 (non binding) I did a full build from source code, created a docker container and did various basic level tests with robotframework based automation and docker-compose based pseudo clusters[1]. Including: * Hdfs federation smoke test * Basic ViewFS configuration * Yarn example jobs * Spa

[jira] [Created] (HDFS-13331) Add lastSeenStateId to RpcRequestHeader.

2018-03-22 Thread Plamen Jeliazkov (JIRA)
Plamen Jeliazkov created HDFS-13331: --- Summary: Add lastSeenStateId to RpcRequestHeader. Key: HDFS-13331 URL: https://issues.apache.org/jira/browse/HDFS-13331 Project: Hadoop HDFS Issue Type

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

2018-03-22 Thread Bharat Viswanadham
+1 (non-binding) 1. verified md5 checksum. 2. Built from source and deployed a 5 node cluster. 3. Ran few basic commands of hdfs operations. 4. Deployed a federated cluster (2 namenodes) and ran few hdfs basic commands 5. Checked NN web UI. Thanks, Bharat On 3/22/18, 10:02 AM, "Ajay Kumar" wro

[jira] [Created] (HDFS-13330) Clean up dead code

2018-03-22 Thread Wei-Chiu Chuang (JIRA)
Wei-Chiu Chuang created HDFS-13330: -- Summary: Clean up dead code Key: HDFS-13330 URL: https://issues.apache.org/jira/browse/HDFS-13330 Project: Hadoop HDFS Issue Type: Improvement

[jira] [Created] (HDFS-13329) Add/ Update disk space counters for trash (trash used, disk remaining etc.)

2018-03-22 Thread Bharat Viswanadham (JIRA)
Bharat Viswanadham created HDFS-13329: - Summary: Add/ Update disk space counters for trash (trash used, disk remaining etc.) Key: HDFS-13329 URL: https://issues.apache.org/jira/browse/HDFS-13329

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Anu Engineer
Hi Lei/Owen, Based on Daryn’s suggestion, we have made HDSL a loadable module inside data node. It relies on the current loadable module support that is already present in HDFS. This module is loaded by Datanodes only when it is configured. --Anu On 3/22/18, 11:25 AM, "Owen O'Malley" wrote

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Owen O'Malley
On Thu, Mar 22, 2018 at 11:09 AM, Lei Xu wrote: > > I have one concrete question about how this HDSL subproject being > separated: Ozone / HDSL was designed in the current way to re-use the > existing HDFS code base as much as possible, thus today for this > container service is in DataNode itsel

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Lei Xu
Hi, Owen Thanks a lot for this proposal, as I believe it has addressed most of the concerns of the community. I have one concrete question about how this HDSL subproject being separated: Ozone / HDSL was designed in the current way to re-use the existing HDFS code base as much as possible, thus t

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Chris Douglas
On Thu, Mar 22, 2018 at 10:23 AM, Andrew Wang wrote: > We want the git hash to match the contents of the tarball and tag, which is > beyond what create release does right now. It doesn't do any git stuff. ...and it can't? Even if this remains a manual step, it's not a significant concession. > I

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Andrew Wang
We want the git hash to match the contents of the tarball and tag, which is beyond what create release does right now. It doesn't do any git stuff. Also sorry if I missed this, but have we gone through all of Owen and Daryn's earlier comments about merge readiness? Some of Daryn's in particular re

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

2018-03-22 Thread Ajay Kumar
+1 (non-binding) - verified signatures and checksums - built from source and setup single node cluster - ran basic hdfs operations - ran TestDFSIO(read/write), wordcount, pi jobs. - basic sanity check of NN UI Thanks, Ajay On 3/22/18, 4:20 AM, "Akira Ajisaka" wrote:

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Chris Douglas
+1 (binding) This compromise seems to address most of the concerns raised during the discussion. Thanks for proposing and driving this, Owen. On Thu, Mar 22, 2018 at 9:30 AM, Andrew Wang wrote: > In Owen's proposal, it says to delete the module from the release branch. > We need to do this since

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Andrew Wang
Hi Anu, Again apologies in advance for phone typing, flight delays means I'm still writing this from an airport :( In Owen's proposal, it says to delete the module from the release branch. We need to do this since the source tarball is our official Apache release artifact, the rest are convenienc

Re: [VOTE] Release Apache Hadoop 3.1.0 (RC0)

2018-03-22 Thread Wangda Tan
Thanks @Bharat for the quick check, the previously staged repository has some issues. I re-deployed jars to nexus. Here's the new repo (1087) https://repository.apache.org/content/repositories/orgapachehadoop-1087/ Other artifacts remain same, no additional code changes. On Wed, Mar 21, 2018 at

[jira] [Created] (HDFS-13328) Abstract ReencryptionHandler recursive logic in separate class.

2018-03-22 Thread Surendra Singh Lilhore (JIRA)
Surendra Singh Lilhore created HDFS-13328: - Summary: Abstract ReencryptionHandler recursive logic in separate class. Key: HDFS-13328 URL: https://issues.apache.org/jira/browse/HDFS-13328 Proje

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2018-03-22 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/728/ [Mar 21, 2018 2:41:19 AM] (rohithsharmaks) YARN-7581. HBase filters are not constructed correctly in ATSv2. [Mar 21, 2018 2:51:35 AM] (yqlin) HDFS-13307. RBF: Improve the use of setQuota command. Contribut

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Mukul Kumar Singh
+1 (non binding) Thanks, Mukul On 22/03/18, 12:31 PM, "Tsz Wo (Nicholas), Sze" wrote: +1 (binding) Tsz-Wo On Wednesday, March 21, 2018, 10:38:03 PM PDT, Jakob Homan wrote: +1 (binding) On 21 March 2018 at 20:12, Shashikant Banerjee wrote: > +1(

Re: [VOTE] Release Apache Hadoop 3.0.1 (RC1)

2018-03-22 Thread Akira Ajisaka
+1 (binding) - verified signatures and checksums - built from source and setup 4 node cluster - ran some examples jobs - verified some HDFS operations Thanks, Akira On 2018/03/22 10:08, Robert Kanter wrote: +1 (binding) - verified signatures and checksums - setup pseudo cluster using Fair Sch

[jira] [Created] (HDFS-13327) Ratis should handle failure conditions where append request to followers can fail

2018-03-22 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created HDFS-13327: Summary: Ratis should handle failure conditions where append request to followers can fail Key: HDFS-13327 URL: https://issues.apache.org/jira/browse/HDFS-13327

Re: [VOTE] Adopt HDSL as a new Hadoop subproject

2018-03-22 Thread Tsz Wo (Nicholas), Sze
+1 (binding) Tsz-Wo On Wednesday, March 21, 2018, 10:38:03 PM PDT, Jakob Homan wrote: +1 (binding) On 21 March 2018 at 20:12, Shashikant Banerjee wrote: > +1(non-binding) > > On 3/21/18, 10:13 AM, "Jitendra Pandey" wrote: > >    +1 (binding) > >    On 3/20/18, 8:39 PM, "Weiwei Yang"