Re: [VOTE] Merge yarn-native-services branch into trunk

2017-11-03 Thread Sunil G
+1 (binding) Tested the branch code and brought up services like sleep and httpd. Also verified UI as well. - Sunil On Tue, Oct 31, 2017 at 1:36 AM Jian He wrote: > Hi All, > > I would like to restart the vote for merging yarn-native-services to trunk. > Since last vote, we have been working

[jira] [Created] (HDFS-12779) [READ] Allow cluster id to be specified to the Image generation tool

2017-11-03 Thread Virajith Jalaparti (JIRA)
Virajith Jalaparti created HDFS-12779: - Summary: [READ] Allow cluster id to be specified to the Image generation tool Key: HDFS-12779 URL: https://issues.apache.org/jira/browse/HDFS-12779 Project:

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-11-03 Thread Eric Yang
+1 (binding) for branch merge. I couldn’t get the patch to work properly though. Yarn-native-services branch works fine. If bugs surface in trunk, we will fix it. Regards, Eric On 11/3/17, 2:45 PM, "Wangda Tan" wrote: Thanks all for the great works! +1 (Binding). Tried to use na

[DISCUSS] A final minor release off branch-2?

2017-11-03 Thread Vinod Kumar Vavilapalli
Hi all, With 3.0.0 GA around the corner (tx for the push, Andrew!), 2.9.0 RC out (tx Arun / Subru!) and 2.8.2 (tx Junping!), I think it's high time we have a discussion on how we manage our developmental bandwidth between 2.x line and 3.x lines. Once 3.0 GA goes out, we will have two parallel

[jira] [Created] (HDFS-12778) [READ] Report multiple locations for PROVIDED blocks

2017-11-03 Thread Virajith Jalaparti (JIRA)
Virajith Jalaparti created HDFS-12778: - Summary: [READ] Report multiple locations for PROVIDED blocks Key: HDFS-12778 URL: https://issues.apache.org/jira/browse/HDFS-12778 Project: Hadoop HDFS

[jira] [Created] (HDFS-12777) [READ] Share URI prefix across Provided blocks to lower memory footprint.

2017-11-03 Thread Virajith Jalaparti (JIRA)
Virajith Jalaparti created HDFS-12777: - Summary: [READ] Share URI prefix across Provided blocks to lower memory footprint. Key: HDFS-12777 URL: https://issues.apache.org/jira/browse/HDFS-12777 Pro

[jira] [Created] (HDFS-12776) [READ] Increasing replication for PROVIDED files should create local replicas

2017-11-03 Thread Virajith Jalaparti (JIRA)
Virajith Jalaparti created HDFS-12776: - Summary: [READ] Increasing replication for PROVIDED files should create local replicas Key: HDFS-12776 URL: https://issues.apache.org/jira/browse/HDFS-12776

[jira] [Created] (HDFS-12775) [READ] Fix capacity reporting for Provided volumes

2017-11-03 Thread Virajith Jalaparti (JIRA)
Virajith Jalaparti created HDFS-12775: - Summary: [READ] Fix capacity reporting for Provided volumes Key: HDFS-12775 URL: https://issues.apache.org/jira/browse/HDFS-12775 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC0)

2017-11-03 Thread Arun Suresh
Hey Vinod, I've cleaned up the RC directory as you requested. Cheers -Arun On Fri, Nov 3, 2017 at 4:09 PM, Vinod Kumar Vavilapalli wrote: > Arun / Subru, > > Thanks for the great work! > > Few quick comments > - Can you cleanup the RC folder to only have tar.gz and src.tar.gz and > their sign

Re: Cutting branch-2.9

2017-11-03 Thread Subru Krishnan
Hello Junping Thanks for taking the time/effort to go through this. Including our reply so that folks in the list have the full context. > 1. It looks like we change from log4j to sel4j (HADOOP-12956) in 2.9 which is a huge incompatible change. Do we have consensus on this in community (as I saw

Re: Cutting branch-2.9

2017-11-03 Thread Junping Du
Below is some findings from my recently run of JACC (https://builds.apache.org/job/Hadoop-2.9-JACC/12/artifact/target/compat-check/report.html) job against 2.9 and 2.8.2. I have discussed with Arun and Subru offline on jdiff report who convinced me some of items are not a big concern. Just tried

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC0)

2017-11-03 Thread Vinod Kumar Vavilapalli
Arun / Subru, Thanks for the great work! Few quick comments - Can you cleanup the RC folder to only have tar.gz and src.tar.gz and their signatures and delete everything else? So that it's easy to pick up the important bits for the voters. For e.g, like this http://people.apache.org/~vinodkv/

[VOTE] Release Apache Hadoop 2.9.0 (RC0)

2017-11-03 Thread Arun Suresh
Hi folks, Apache Hadoop 2.9.0 is the first stable release of Hadoop 2.9 line and will be the latest stable/production release for Apache Hadoop - it includes 30 New Features with 500+ subtasks, 407 Improvements, 787 Bug fixes new fixed issues since 2.8.2 . More information about the 2.

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2017-11-03 Thread Vinod Kumar Vavilapalli
> At a minimum, it should at least be using it’s own maven module for a > lot of the bits that generates it’s own maven jars so that we can split this > functionality up at build/test time. I expected this to be the case, but looks like it isn't. There's lot of value in splitting the HDF

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-11-03 Thread Wangda Tan
Thanks all for the great works! +1 (Binding). Tried to use native services to build and run applications successfully. - Wangda On Fri, Nov 3, 2017 at 11:50 AM, Arun Suresh wrote: > +1 (binding) > > Cheers > -Arun > > On Nov 3, 2017 11:44 AM, "Chandni Singh" wrote: > > > +1 > > Thanks, > > Ch

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2017-11-03 Thread Ravi Prakash
Hi folks! Thank you for sharing the design docs and the tremendous amount of work that has gone into Ozone. I'm grateful that atleast someone is trying to drastically improve HDFS. *If* there is a meeting to discuss this merge, could I please also be invited? Have we ever thought about distribut

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2017-11-03 Thread sanjay Radia
Konstantine, Thanks for your comments, questions and feedback. I have attached a document to the HDFS-7240 jira that explains a design for scaling HDFS and how Ozone paves the way towards the full solution. https://issues.apache.org/jira/secure/attachment/12895963/HDFS%20Scalability%20and%2

[jira] [Created] (HDFS-12774) Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client

2017-11-03 Thread Nanda kumar (JIRA)
Nanda kumar created HDFS-12774: -- Summary: Ozone: OzoneClient: Moving OzoneException from hadoop-hdfs to hadoop-hdfs-client Key: HDFS-12774 URL: https://issues.apache.org/jira/browse/HDFS-12774 Project: H

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2017-11-03 Thread Allen Wittenauer
> On Nov 3, 2017, at 12:08 PM, Stack wrote: > > On Sat, Oct 28, 2017 at 2:00 PM, Konstantin Shvachko > wrote: > >> It is an interesting question whether Ozone should be a part of Hadoop. > > I don't see a direct answer to this question. Is there one? Pardon me if > I've not seen it but I'm in

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2017-11-03 Thread Xiaoyu Yao
Hi Lei, Thank you for your interest in Ozone. Let me answer each of the specific questions. > As the current state of Ozone implementation, what are the major > benefits of using today’s Ozone over HDFS? Scale - HDFS tops out at 500/700 million keys; Ozone primary use case is to go beyond tha

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

2017-11-03 Thread Stack
On Sat, Oct 28, 2017 at 2:00 PM, Konstantin Shvachko wrote: > Hey guys, > > It is an interesting question whether Ozone should be a part of Hadoop. > I don't see a direct answer to this question. Is there one? Pardon me if I've not seen it but I'm interested in the response. I ask because IMO

Re: [VOTE] Merge yarn-native-services branch into trunk

2017-11-03 Thread Arun Suresh
+1 (binding) Cheers -Arun On Nov 3, 2017 11:44 AM, "Chandni Singh" wrote: > +1 > Thanks, > Chandni > > From: Jian He > Sent: Monday, October 30, 2017 1:49 PM > To: yarn-...@hadoop.apache.org; Hdfs-dev; Hadoop Common; > mapreduce-...@hadoop.apache.org >

[jira] [Created] (HDFS-12773) RBF: Improve State Store FS implementation

2017-11-03 Thread JIRA
Íñigo Goiri created HDFS-12773: -- Summary: RBF: Improve State Store FS implementation Key: HDFS-12773 URL: https://issues.apache.org/jira/browse/HDFS-12773 Project: Hadoop HDFS Issue Type: Sub-ta

[jira] [Created] (HDFS-12772) RBF: Track Router states

2017-11-03 Thread JIRA
Íñigo Goiri created HDFS-12772: -- Summary: RBF: Track Router states Key: HDFS-12772 URL: https://issues.apache.org/jira/browse/HDFS-12772 Project: Hadoop HDFS Issue Type: Sub-task Rep

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-11-03 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/579/ [Nov 3, 2017 12:15:33 AM] (Arun Suresh) HADOOP-15013. Fix ResourceEstimator findbugs issues. (asuresh) [Nov 3, 2017 12:39:23 AM] (subru) YARN-7432. Fix DominantResourceFairnessPolicy serializable findbugs

[jira] [Created] (HDFS-12771) Add genstamp and block size to metasave Corrupt blocks list

2017-11-03 Thread Kuhu Shukla (JIRA)
Kuhu Shukla created HDFS-12771: -- Summary: Add genstamp and block size to metasave Corrupt blocks list Key: HDFS-12771 URL: https://issues.apache.org/jira/browse/HDFS-12771 Project: Hadoop HDFS

[jira] [Resolved] (HDFS-3821) Backport HDFS-3626 to branch-1 (Creating file with invalid path can corrupt edit log)

2017-11-03 Thread Andras Bokor (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor resolved HDFS-3821. Resolution: Won't Fix > Backport HDFS-3626 to branch-1 (Creating file with invalid path can corrupt

[jira] [Created] (HDFS-12770) Add doc about how to disable client socket cache

2017-11-03 Thread Weiwei Yang (JIRA)
Weiwei Yang created HDFS-12770: -- Summary: Add doc about how to disable client socket cache Key: HDFS-12770 URL: https://issues.apache.org/jira/browse/HDFS-12770 Project: Hadoop HDFS Issue Type: