Apache Hadoop qbt Report: trunk+JDK9 on Linux/x86

For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java9-linux-x86/2/ [Oct 20, 2017 9:24:17 PM] (haibochen) YARN-7372. [Oct 20, 2017 9:27:04 PM] (stevel) HADOOP-14942. DistCp#cleanup() should check whether jobFS is null. [Oct 20, 2017 10:54:15 PM] (wangda) YARN-7318. Fix shell

Re: [DISCUSS] Feature Branch Merge and Security Audits

Hi Eric - Thanks for the additional item suggestions! "We might want to start a security section for Hadoop wiki for each of the services and components. This helps to track what has been completed." Do you mean to keep the audit checklist for each service and component there? Interesting idea,

2.9.0 status update (10/20/2017)

Today was the feature freeze date and we are glad to inform that all the major planned features are merged in branch-2: https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#Roadmap-Plannedfeatures : Kudos to everyone who pulled through multiple blockers and made this happen. Special shoutout

Re: [DISCUSS] Feature Branch Merge and Security Audits

The check list looks good. Some more items to add: Kerberos TGT renewal SPNEGO support Delegation token Proxy User ACL CVE tracking list We might want to start a security section for Hadoop wiki for each of the services and components. This helps to track what has been completed. How do

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

All - I broke this list of questions out into a separate DISCUSS thread where we can iterate over how a security audit process at merge time might look and whether it is even something that we want to take on. I will try and continue discussion on that thread and drive that to some conclusion bef

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Thanks, Junping. +1 (non-binding) - Build from hadoop-2.8.2-src.tar.gz (skipped test) and also tried hadoop-2.8.2.tar.gz - Setup a 30-node HDFS cluster - Run some basic HDFS/webhdfs commands (read/write/mkdirs/listStatus/getFileInfo) - Check NN Web UI and operations through the UI - Run a simple

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Hi Junping, Thanks for preparing the 2.8.2-RC1 release. Verified the following: - Built from source on Mac OS X 10.11.6 with Java 1.7.0_79 - Deployed binary to a 3-node docker cluster - Sanity checks - Basic dfs operations - MapReduce Wordcount & Grep +1 (non-binding) Thanks,

2017-10-20 Hadoop 3 release status update

https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+3+release+status+updates 2017-10-20 Apologies for skipping the update last week. Here's how we're tracking for GA. Highlights: - Merge of HDFS router-based federation and API-based scheduler configuration with no reported problems.

Re: [DISCUSS] Feature Branch Merge and Security Audits

Adding security@hadoop list as well... On Fri, Oct 20, 2017 at 2:29 PM, larry mccay wrote: > All - > > Given the maturity of Hadoop at this point, I would like to propose that > we start doing explicit security audits of features at merge time. > > There are a few reasons that I think this is a

[DISCUSS] Feature Branch Merge and Security Audits

All - Given the maturity of Hadoop at this point, I would like to propose that we start doing explicit security audits of features at merge time. There are a few reasons that I think this is a good place/time to do the review: 1. It represents a specific snapshot of where the feature stands as a

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

I previously sent this same email from my work email and it doesn't seem to have gone through - resending from apache account (apologizing up from for the length) For such sizable merges in Hadoop, I would like to start doing security audits in order to have an initial idea of the attack surfa

[jira] [Resolved] (HDFS-12688) HDFS File Not Removed Despite Successful "Moved to .Trash" Message

[ https://issues.apache.org/jira/browse/HDFS-12688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shriya Gupta resolved HDFS-12688. - Resolution: Not A Bug Another job was recreating the file that the user suspected wasn't being de

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

Hi Steve, In addition to everything Weiwei mentioned (chapter 3 of user guide), if you really want to drill down to REST protocol you might want to apply this patch and build ozone. https://issues.apache.org/jira/browse/HDFS-12690 This will generate an Open API (https://www.openapis.org , http

答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

Hi Steve The code is available in HDFS-7240 feature branch, public git repo here. I am not sure if there is a "public" API for object stores, but the design doc uses m

[jira] [Created] (HDFS-12692) Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key

Elek, Marton created HDFS-12692: --- Summary: Ozone: fix javadoc/unit test for calling MetadataStore.getRangeKVs with non existing key Key: HDFS-12692 URL: https://issues.apache.org/jira/browse/HDFS-12692

Re: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

Wow, big piece of work 1. Where is a PR/branch on github with rendered docs for us to look at? 2. Have you made any public APi changes related to object stores? That's probably something I'll have opinions on more than implementation details. thanks > On 19 Oct 2017, at 02:54, Yang Weiwei wro

[jira] [Created] (HDFS-12691) Ozone: Decrease interval time of SCMBlockDeletingService for improving the efficiency

Yiqun Lin created HDFS-12691: Summary: Ozone: Decrease interval time of SCMBlockDeletingService for improving the efficiency Key: HDFS-12691 URL: https://issues.apache.org/jira/browse/HDFS-12691 Project:

[jira] [Created] (HDFS-12690) Ozone: generate swagger descriptor for the Ozone REST Api

Elek, Marton created HDFS-12690: --- Summary: Ozone: generate swagger descriptor for the Ozone REST Api Key: HDFS-12690 URL: https://issues.apache.org/jira/browse/HDFS-12690 Project: Hadoop HDFS I