Re: Block invalid IOException causes the DFSClient domain socket being disabled

2017-10-25 Thread John Zhuge
Hi Xie Gang, Thanks for reporting the issue. How was the block invalidated on DN? I recently created a unit test that might trigger a somewhat similar symptom: 1. Read a 4000-byte file in MiniDFS. It should go thru domain socket. 2. Call input stream unbuffer. It will close the block reade

Block invalid IOException causes the DFSClient domain socket being disabled

2017-10-25 Thread Xie Gang
Hi, We use HDFS2.4 & 2.6, and recently hit a issue that DFSClient domain socket is disabled when datanode throw block invalid exception. The block is invalidated for some reason on datanote and it's OK. Then DFSClient tries to access this block on this datanode via domain socket. This triggers a

[RESULT] [VOTE] Release Apache Hadoop 2.8.2 (RC1)

2017-10-25 Thread Junping Du
??Thanks again for all who verified and voted! I give my binding +1 to conclude the vote for 2.8.2 RC1, based on: - Build from source and verify signatures - Deploy pseudo-distributed cluster and run some simple job, like: pi, sleep, etc. - Verify UI of daemons, like: NameNode, ResourceManager, N

[jira] [Created] (HDFS-12714) Hadoop 3 missing fix for HDFS-5169

2017-10-25 Thread Joe McDonnell (JIRA)
Joe McDonnell created HDFS-12714: Summary: Hadoop 3 missing fix for HDFS-5169 Key: HDFS-12714 URL: https://issues.apache.org/jira/browse/HDFS-12714 Project: Hadoop HDFS Issue Type: Bug

[jira] [Created] (HDFS-12713) [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata

2017-10-25 Thread Virajith Jalaparti (JIRA)
Virajith Jalaparti created HDFS-12713: - Summary: [READ] Refactor FileRegion and BlockAliasMap to separate out HDFS metadata and PROVIDED storage metadata Key: HDFS-12713 URL: https://issues.apache.org/jira/bro

[DISCUSS] Merging libhdfs++ (HDFS-8707) into trunk

2017-10-25 Thread Jim Clampffer
Hi everyone, I'd like to start a thread to discuss merging the HDFS-8707 aka libhdfs++ into trunk (as a beta release). libhdfs++ is an HDFS client written in C++ designed to be used in applications that are written in non-JVM based languages. In its current state it supports kerberos authenticat

[jira] [Created] (HDFS-12712) [9806] Code style cleanup

2017-10-25 Thread JIRA
Íñigo Goiri created HDFS-12712: -- Summary: [9806] Code style cleanup Key: HDFS-12712 URL: https://issues.apache.org/jira/browse/HDFS-12712 Project: Hadoop HDFS Issue Type: Sub-task Re

Re: [DISCUSS] Feature Branch Merge and Security Audits

2017-10-25 Thread larry mccay
Terrific additions, Mike! I will spin a new revision and incorporate your additions. #8 is a great topic - given that Hadoop is insecure by default. Actual movement to Secure by Default would be a challenge both technically (given the need for kerberos) and discussion-wise. Asking whether you have

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2017-10-25 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/569/ [Oct 25, 2017 5:17:56 AM] (haibochen) YARN-7389. Make TestResourceManager Scheduler agnostic. (Robert Kanter [Oct 25, 2017 8:17:21 AM] (aajisaka) HADOOP-14030. PreCommit TestKDiag failure. Contributed by W

[jira] [Resolved] (HDFS-4493) webhdfs fails after SPNEGO token expires

2017-10-25 Thread Daryn Sharp (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp resolved HDFS-4493. --- Resolution: Works for Me Release Note: This issue was fixed years ago when authenticated url was

[jira] [Created] (HDFS-12711) deadly hdfs test

2017-10-25 Thread Allen Wittenauer (JIRA)
Allen Wittenauer created HDFS-12711: --- Summary: deadly hdfs test Key: HDFS-12711 URL: https://issues.apache.org/jira/browse/HDFS-12711 Project: Hadoop HDFS Issue Type: Test Repor

[jira] [Created] (HDFS-12710) HTTPFS HTTP max header size env variable is not respected in branch-2

2017-10-25 Thread Istvan Fajth (JIRA)
Istvan Fajth created HDFS-12710: --- Summary: HTTPFS HTTP max header size env variable is not respected in branch-2 Key: HDFS-12710 URL: https://issues.apache.org/jira/browse/HDFS-12710 Project: Hadoop HDF

[jira] [Created] (HDFS-12709) NFS gateway NoSuchFieldError: LOG

2017-10-25 Thread liu huan (JIRA)
liu huan created HDFS-12709: --- Summary: NFS gateway NoSuchFieldError: LOG Key: HDFS-12709 URL: https://issues.apache.org/jira/browse/HDFS-12709 Project: Hadoop HDFS Issue Type: Bug Repor