[jira] [Created] (HDFS-12823) Backport HDFS-9259 "Make SO_SNDBUF size configurable at DFSClient" to branch-2.7

2017-11-15 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-12823: -- Summary: Backport HDFS-9259 "Make SO_SNDBUF size configurable at DFSClient" to branch-2.7 Key: HDFS-12823 URL: https://issues.apache.org/jira/browse/HDFS-12823 Project: H

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Brahma Reddy Battula
+1 ( non-binding) -Built from the source -Installed 3 Node HA cluster and a pseudo cluster -Verified through hdfs shell commands -Verified HDFS router federation -Ran sample jobs like pi,Slive --Brahma Reddy On Tue, Nov 14, 2017 at 5:40 AM, Arun Suresh wrote: > Hi Folks, > > Apache Hadoop 2

[jira] [Created] (HDFS-12822) HDFS unit test failure in AArch64. TestDirectoryScanner.testThrottling: Throttle is too permissive

2017-11-15 Thread Guangming Zhang (JIRA)
Guangming Zhang created HDFS-12822: -- Summary: HDFS unit test failure in AArch64. TestDirectoryScanner.testThrottling: Throttle is too permissive Key: HDFS-12822 URL: https://issues.apache.org/jira/browse/HDFS-128

[jira] [Reopened] (HDFS-12681) Fold HdfsLocatedFileStatus into HdfsFileStatus

2017-11-15 Thread Chris Douglas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Douglas reopened HDFS-12681: -- Reopening to revert this, and try another approach > Fold HdfsLocatedFileStatus into HdfsFileStatus

[jira] [Created] (HDFS-12821) Block invalid IOException causes the DFSClient domain socket being disabled

2017-11-15 Thread Gang Xie (JIRA)
Gang Xie created HDFS-12821: --- Summary: Block invalid IOException causes the DFSClient domain socket being disabled Key: HDFS-12821 URL: https://issues.apache.org/jira/browse/HDFS-12821 Project: Hadoop HDFS

Decommissioned datanode is counted in service cause datanode allcating failure

2017-11-15 Thread Xie Gang
Hi, When allocate a datanode when dfsclient write with considering the load, it checks if the datanode is overloaded by calculating the average xceivers of all the in service datanode. But if the datanode is decommissioned and become dead, it's still treated as in service, which make the average l

[jira] [Created] (HDFS-12820) Decommissioned datanode is counted in service cause datanode allcating failure

2017-11-15 Thread Gang Xie (JIRA)
Gang Xie created HDFS-12820: --- Summary: Decommissioned datanode is counted in service cause datanode allcating failure Key: HDFS-12820 URL: https://issues.apache.org/jira/browse/HDFS-12820 Project: Hadoop HD

[jira] [Created] (HDFS-12819) Setting/Unsetting EC policy shows warning if the directory is not empty

2017-11-15 Thread Lei (Eddy) Xu (JIRA)
Lei (Eddy) Xu created HDFS-12819: Summary: Setting/Unsetting EC policy shows warning if the directory is not empty Key: HDFS-12819 URL: https://issues.apache.org/jira/browse/HDFS-12819 Project: Hadoop

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread IƱigo Goiri
+1 (non-binding) Deployed in a cluster with 48 nodes and 8 subclusters: - YARN federation - HDFS Router-based federation - Yarn UI 2 Executed a few Pi job using both HDFS and YARN federation. Everything worked correctly. The YARN UI 2 showed the jobs, etc. On Wed, Nov 15, 2017 at 2:05

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Chris Douglas
+1 (binding) Verified source tarball. Checksum and signature match, built from source, ran some unit tests. Skimmed NOTICE/LICENSE. -C On Mon, Nov 13, 2017 at 4:10 PM, Arun Suresh wrote: > Hi Folks, > > Apache Hadoop 2.9.0 is the first release of Hadoop 2.9 line and will be the > starting releas

Re: [DISCUSS] A final minor release off branch-2?

2017-11-15 Thread Andrew Purtell
> From recent classpath isolation work, I was surprised to find out that many of our downstream projects (HBase, Tez, etc.) are still consuming many non-public, server side APIs of Hadoop, not saying the projects/products outside of hadoop ecosystem. Our API compatibility test does not (and should

Re: [DISCUSS] A final minor release off branch-2?

2017-11-15 Thread Ray Chiang
On 11/15/17 10:34 AM, Andrew Wang wrote: Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: 3. Beside incompatibilities, there is also possible to have performance regressions (lower throughput, higher latency, slower job running, bigger memory footprint or even memory leaking, etc

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Kuhu Shukla
+1 (non-binding) 1. Built from source. 2. Deployed to pseudo node cluster 3. Ran sample jobs. Regards, Kuhu On Wed, Nov 15, 2017 at 1:07 PM, Eric Badger wrote: > +1 (non-binding) > > - Verified all hashes and checksums > - Built from source on macOS 10.12.6, Java 1.8.0u65 > - Deployed a pseud

Re: [DISCUSS] A final minor release off branch-2?

2017-11-15 Thread Steve Loughran
On 15 Nov 2017, at 09:37, Junping Du mailto:j...@hortonworks.com>> wrote: 2. From recent classpath isolation work, I was surprised to find out that many of our downstream projects (HBase, Tez, etc.) are still consuming many non-public, server side APIs of Hadoop, not saying the projects/prod

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Eric Badger
+1 (non-binding) - Verified all hashes and checksums - Built from source on macOS 10.12.6, Java 1.8.0u65 - Deployed a pseudo cluster - Ran some example jobs Thanks, Eric On Wed, Nov 15, 2017 at 10:24 AM, Carlo Aldo Curino wrote: > +1 (binding) > > On Nov 15, 2017 8:23 AM, "Mukul Kumar Singh"

Re: [DISCUSS] A final minor release off branch-2?

2017-11-15 Thread Andrew Wang
Hi Junping, On Wed, Nov 15, 2017 at 1:37 AM, Junping Du wrote: > Thanks Vinod to bring up this discussion, which is just in time. > > I agree with most responses that option C is not a good choice as our > community bandwidth is precious and we should focus on very limited > mainstream branches

[jira] [Resolved] (HDFS-12817) Support multiple storages in DataNodeCluster / SimulatedFSDataset

2017-11-15 Thread Jason Lowe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe resolved HDFS-12817. --- Resolution: Duplicate > Support multiple storages in DataNodeCluster / SimulatedFSDataset > --

[jira] [Created] (HDFS-12818) Support multiple storages in DataNodeCluster / SimulatedFSDataset

2017-11-15 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-12818: -- Summary: Support multiple storages in DataNodeCluster / SimulatedFSDataset Key: HDFS-12818 URL: https://issues.apache.org/jira/browse/HDFS-12818 Project: Hadoop HDFS

[jira] [Created] (HDFS-12817) Support multiple storages in DataNodeCluster / SimulatedFSDataset

2017-11-15 Thread Erik Krogen (JIRA)
Erik Krogen created HDFS-12817: -- Summary: Support multiple storages in DataNodeCluster / SimulatedFSDataset Key: HDFS-12817 URL: https://issues.apache.org/jira/browse/HDFS-12817 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Carlo Aldo Curino
+1 (binding) On Nov 15, 2017 8:23 AM, "Mukul Kumar Singh" wrote: > +1 (non-binding) > > I built from source on Mac OS X 10.13.1 Java 1.8.0_111 > > - Deployed on a single node cluster. > - Deployed a ViewFS cluster with two hdfs mount points. > - Performed basic sanity checks. > - Performed DFS o

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Mukul Kumar Singh
+1 (non-binding) I built from source on Mac OS X 10.13.1 Java 1.8.0_111 - Deployed on a single node cluster. - Deployed a ViewFS cluster with two hdfs mount points. - Performed basic sanity checks. - Performed DFS operations(put, ls, mkdir, touch) Thanks, Mukul > On 14-Nov-2017, at 5:40 AM, Ar

[jira] [Resolved] (HDFS-12803) We should not lock FsNamesystem even we operate a sub directory, we should refinement the lock

2017-11-15 Thread Erik Krogen (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-12803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Erik Krogen resolved HDFS-12803. Resolution: Duplicate This is a longstanding request tracked in HDFS-5453 > We should not lock FsNa

[jira] [Created] (HDFS-12816) Fix TestBalancerRPCDelay#testBalancerRPCDelay

2017-11-15 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created HDFS-12816: Summary: Fix TestBalancerRPCDelay#testBalancerRPCDelay Key: HDFS-12816 URL: https://issues.apache.org/jira/browse/HDFS-12816 Project: Hadoop HDFS Iss

[jira] [Resolved] (HDFS-4262) Backport HTTPFS to Branch 1

2017-11-15 Thread Andras Bokor (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andras Bokor resolved HDFS-4262. Resolution: Won't Fix > Backport HTTPFS to Branch 1 > --- > >

[jira] [Created] (HDFS-12815) Fix TestUnbuffer#testUnbufferException because of expected error message mismatch

2017-11-15 Thread Mukul Kumar Singh (JIRA)
Mukul Kumar Singh created HDFS-12815: Summary: Fix TestUnbuffer#testUnbufferException because of expected error message mismatch Key: HDFS-12815 URL: https://issues.apache.org/jira/browse/HDFS-12815

Re: [DISCUSS] A final minor release off branch-2?

2017-11-15 Thread Junping Du
Thanks Vinod to bring up this discussion, which is just in time. I agree with most responses that option C is not a good choice as our community bandwidth is precious and we should focus on very limited mainstream branches to develop, test and deployment. Of course, we should still follow Apache

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Sunil G
+1 (binding) Built from source. - Tested few cases in an HA cluster and tried to do failover by using rmadmin commands etc. This seems works fine including submitting apps. - I also tested many MR apps and all are running fine w/o any issues. - Majorly tested below feature sanity to

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

2017-11-15 Thread Rohith Sharma K S
+1(binding) * Downloaded source and verified for checksum. Built from source and deployed RM HA Non-Secure cluster along with Atsv2 and yarn-ui enabled! * Tested for ** RM HA/Restart/Work-preserving restart and NM work preserving restart scenarios ** Application priority and Ap