Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Andrew Wang
I think I understand a bit better, though now I ask how this date is different from the release date. Based on the HowToRelease instructions, we set the release date to when the release vote passes. So, start of release vote vs. end of release vote doesn't seem that different, and these dates are s

[DISCUSS] Retire BKJM from trunk?

2016-07-26 Thread Vinayakumar B
Hi All, BKJM was Active and made much stable when the NameNode HA was implemented and there was no QJM implemented. Now QJM is present and is much stable which is adopted by many production environment. I wonder whether it would be a good time to retire BKJM from trunk? Are there an

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Tsuyoshi Ozawa
> Andrew: I bet many would assume it's the release date, like how Ubuntu releases are numbered. Good point. Maybe I confuse you because of lack of explanation. I assume that "branch-cut off timing" mean the timing of freezing branch like when starting the release vote. It's because that the relea

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Andrew Wang
Thanks for replies Akira and Tsuyoshi, inline: Akira: Assuming 3.0.0-alpha1 will be released between 2.7.0 and 2.8.0, we > need to add 3.0.0-alphaX if 2.8.0 is in the fix versions of a jira and we > don't need to add 3.0.0-alphaX if 2.7.0 is in the fix versions of a jira. > Is it right? Yes, cor

[DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Tsuyoshi Ozawa
Hi Vinod, Thanks all guys for starting discussion! My suggestion is adding the date when branch cut is done: like 3.0.0-alpha1-20160724, 2.8.0-20160730 or something. Pros:-) It's totally ordered. If we have a policy such as backporting to maintainance branches after the date, users can find that

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Akira Ajisaka
Thanks Vinod and Andrew for the summary. > Here's an attempt at encoding this policy as a set of rules for setting fix > versions (credit to Allen): > > 1) Set the fix version for all a.b.c versions, where c > 0. > 2) For each major release line, set the lowest a.b.0 version. Assuming 3.0.0-al

[jira] [Created] (HDFS-10695) When using NN Fedration,DirectoryScanner throw IllegalStateException

2016-07-26 Thread shenxianqiang (JIRA)
shenxianqiang created HDFS-10695: Summary: When using NN Fedration,DirectoryScanner throw IllegalStateException Key: HDFS-10695 URL: https://issues.apache.org/jira/browse/HDFS-10695 Project: Hadoop HD

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Akira Ajisaka
+1 for the source tarball. - Downloaded source tarball and binary tarball - Verified signatures and checksums - Compiled and built a single node cluster - Compiled Hive 2.1.0/1.2.1 and Tez 0.8.4/0.7.1 using Hadoop 2.7.3 pom successfully - Ran some Hive on Tez queries successfully Thanks, Akira

[jira] [Created] (HDFS-10694) BlockManager.processReport() should print blockReportId in each log message.

2016-07-26 Thread Konstantin Shvachko (JIRA)
Konstantin Shvachko created HDFS-10694: -- Summary: BlockManager.processReport() should print blockReportId in each log message. Key: HDFS-10694 URL: https://issues.apache.org/jira/browse/HDFS-10694

[jira] [Created] (HDFS-10693) metaSave should print blocks, not LightWeightHashSet

2016-07-26 Thread Konstantin Shvachko (JIRA)
Konstantin Shvachko created HDFS-10693: -- Summary: metaSave should print blocks, not LightWeightHashSet Key: HDFS-10693 URL: https://issues.apache.org/jira/browse/HDFS-10693 Project: Hadoop HDFS

[jira] [Reopened] (HDFS-4176) EditLogTailer should call rollEdits with a timeout

2016-07-26 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu reopened HDFS-4176: - it was not related to HDFS-9659. > EditLogTailer should call rollEdits with a timeout > --

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Eric Payne
+1 (non-binding) Thanks, Vinod, for all of your hard work and congratulations in completing this release. After downloading and building the source, I installed Hadoop 2.7.3 RC0 on a 3-node, multi-tenant, insecure cluster. I ran manual tests to ensure the following: - Ensure that user limit perc

[jira] [Resolved] (HDFS-4176) EditLogTailer should call rollEdits with a timeout

2016-07-26 Thread Lei (Eddy) Xu (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lei (Eddy) Xu resolved HDFS-4176. - Resolution: Duplicate HDFS-9659 adds timeout to RPC calls, thus fixed this issue. > EditLogTailer

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Kuhu Shukla
+1 (non-binding). - Built Tar ball from source. - Deployed pseudo-distributed cluster. - Ran sleep job. Thank you Vinod! Regards,Kuhu Shukla On Tuesday, July 26, 2016 4:17 PM, Andrew Wang wrote: On Tue, Jul 26, 2016 at 1:23 PM, Karthik Kambatla wrote: > IIRR, the vote is o

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Andrew Wang
On Tue, Jul 26, 2016 at 1:23 PM, Karthik Kambatla wrote: > IIRR, the vote is on source artifacts and binaries are for convenience. > > Let me refine this statement a bit. Both the binary tarball and the JARs we publish to Maven are still official release artifacts. This is why we need L&Ns for th

Re: [DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Andrew Wang
Thanks Vinod for forking the thread. Let me try and summarize what Allen and I talked about in the previous thread. Currently, we've been marking JIRAs with fix versions of both 2.6.x and 2.7.x. IIUC, the chronological ordering between these two lines is actually not important. If you're on 2.6.1,

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Eric Badger
* Verified mds and pgp signatures of both source and binary* Built tarball from source on OS X 10.11.6 (El Capitan)* Deployed in pseudo-distributed mode* Ran sleep jobs and other randomly selected tests on both MapReduce and Tez* Visually verified the RM and history server UIs  Thanks, Eric

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Karthik Kambatla
IIRR, the vote is on source artifacts and binaries are for convenience. If that is right, I am open to either options - do another RC or continue this vote and fix the binary artifacts. On Tue, Jul 26, 2016 at 12:11 PM, Vinod Kumar Vavilapalli < vino...@apache.org> wrote: > Thanks Daniel and Wei

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Naganarasimha Garla
Tested as follows - deployed a pseudo cluster from RC0 tar - Verified signature and checksum - Run MR sleep on YARN - With/Without node labels enabled. - Verified ATS V1 (One small issue though not regression, tracking URL for running app is shown as Unassigned, will check and raise if required )

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Vinod Kumar Vavilapalli
But, everyone please do continue your sanity checking on RC0 in case there are more issues to be fixed. Thanks +Vinod > On Jul 26, 2016, at 12:11 PM, Vinod Kumar Vavilapalli > wrote: > > Thanks Daniel and Wei. > > I think these are worth fixing, I’m withdrawing this RC. Will look at fixing

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Vinod Kumar Vavilapalli
Thanks Daniel and Wei. I think these are worth fixing, I’m withdrawing this RC. Will look at fixing these issues and roll a new candidate with the fixes as soon as possible. Thanks +Vinod > On Jul 26, 2016, at 11:05 AM, Wei-Chiu Chuang wrote: > > I noticed two issues: > > (1) I ran hadoop ch

[DISCUSS] Release numbering semantics with concurrent (>2) releases [Was Setting JIRA fix versions for 3.0.0 releases]

2016-07-26 Thread Vinod Kumar Vavilapalli
Forking the thread to make sure it attracts enough eye-balls. The earlier one was about 3.0.0 specifically and I don’t think enough people were watching that. I’ll try to summarize a bit. # Today’s state of release numbering and ordering: So far, all the releases we have done, we have follow

[jira] [Reopened] (HDFS-10543) hdfsRead read stops at block boundary

2016-07-26 Thread James Clampffer (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Clampffer reopened HDFS-10543: Assignee: James Clampffer Reopening this. Recently found some data being mangled that alw

[jira] [Created] (HDFS-10692) Point JDiff base version for HDFS from 2.6.0 to 2.7.3

2016-07-26 Thread Wangda Tan (JIRA)
Wangda Tan created HDFS-10692: - Summary: Point JDiff base version for HDFS from 2.6.0 to 2.7.3 Key: HDFS-10692 URL: https://issues.apache.org/jira/browse/HDFS-10692 Project: Hadoop HDFS Issue Typ

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Wei-Chiu Chuang
I noticed two issues: (1) I ran hadoop checknative, but it seems the binary tarball was not compiled with native library for Linux. On the contrary, the Hadoop built from source tarball with maven -Pnative can find the native libraries on the same host. (2) I noticed that the release dates in CHA

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Daniel Templeton
I just downloaded the build tarball and deployed it on a 2-node cluster. It looks to me like it's compiled for the wrong platform: # file /usr/lib/hadoop/bin/container-executor /usr/lib/hadoop/bin/container-executor: setuid setgid Mach-O 64-bit executable I'm also seeing the no-native-librar

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Rushabh Shah
Thanks Vinod for all the release work ! +1 (non-binding). * Downloaded from source and built it.* Deployed a pseudo distributed cluster. * Ran some sample jobs: sleep, pi* Ran some dfs commands.* Everything works fine. On Friday, July 22, 2016 9:16 PM, Vinod Kumar Vavilapalli wrote: H

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Masatake Iwasaki
Thanks for putting this up, Vinod. +1 (non-binding) * verified signature and mds of source and binary tarball * built from source tarball on CentOS 6 * built site documentation * deployed 3-node cluster with NN-HA and RM-HA, ran example jobs * built rpms by using Bigtop, deployed 3-node cluster

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

2016-07-26 Thread Apache Jenkins Server
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/114/ [Jul 25, 2016 1:45:03 PM] (stevel) HADOOP-13406 S3AFileSystem: Consider reusing filestatus in delete() and [Jul 25, 2016 2:50:23 PM] (stevel) HADOOP-13188 S3A file-create should throw error rather than ove

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Rohith Sharma K S
+1(non-binding) Downloaded and built from source Cluster installed in 3 nodes and verified running simple MR jobs. Verified for RM HA , RM work preserving restart with CapacityScheduler Thanks & Regards Rohith Sharma K S > On Jul 26, 2016, at 6:50 PM, Vinayakumar B wrote: > > +1 (binding) >

[jira] [Reopened] (HDFS-10668) Fix intermittently failing UT TestDataNodeMXBean#testDataNodeMXBeanBlockCount

2016-07-26 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran reopened HDFS-10668: --- beaking the build I'm afraid. From jenkins {code} [ERROR] Failed to execute goal org.apache.maven

RE: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Vinayakumar B
+1 (binding) 1. Downloaded and Built from branch-2.7.3 2. Started up HDFS and YARN in Single Node cluster. 3. Ran WordCount job multiple times and Success. 4. Verified the "Release Notes" available at the URL mentioned by Vinod. Apart from that, Faced same issues as Andrew wang, while running t

Re: [VOTE] Release Apache Hadoop 2.7.3 RC0

2016-07-26 Thread Rakesh Radhakrishnan
Thank you Vinod. +1 (non-binding) - downloaded and built from source - deployed HDFS-HA cluster and tested few switching behaviors - executed few hdfs commands from command line - viewed basic UI - ran HDFS/Common unit tests - checked LICENSE and NOTICE files Regards, Rakesh Intel On Tue, Jul 2

[jira] [Created] (HDFS-10691) FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException

2016-07-26 Thread Yiqun Lin (JIRA)
Yiqun Lin created HDFS-10691: Summary: FileDistribution fails in hdfs oiv command due to ArrayIndexOutOfBoundsException Key: HDFS-10691 URL: https://issues.apache.org/jira/browse/HDFS-10691 Project: Hadoo

[jira] [Created] (HDFS-10690) Optimize insertion/removal of replica in ShortCircuitCache.java

2016-07-26 Thread Fenghua Hu (JIRA)
Fenghua Hu created HDFS-10690: - Summary: Optimize insertion/removal of replica in ShortCircuitCache.java Key: HDFS-10690 URL: https://issues.apache.org/jira/browse/HDFS-10690 Project: Hadoop HDFS