Please ignore this reply, was supposed to be a nudge on internal lists. +Vinod
On Nov 11, 2015, at 1:15 PM, Vinod Vavilapalli <vino...@hortonworks.com<mailto:vino...@hortonworks.com>> wrote: Started a thread on 2.8.0. Please give feedback etc. I’ll need help from both the teams to get this done right. +Vinod Begin forwarded message: From: Vinod Kumar Vavilapalli <vino...@hortonworks.com<mailto:vino...@hortonworks.com>> Subject: Re: [DISCUSS] Looking to a 2.8.0 release Date: November 11, 2015 at 12:13:54 PM PST To: Hadoop Common <common-...@hadoop.apache.org<mailto:common-...@hadoop.apache.org>>, "yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org>" <yarn-...@hadoop.apache.org<mailto:yarn-...@hadoop.apache.org>>, "hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>" <hdfs-dev@hadoop.apache.org<mailto:hdfs-dev@hadoop.apache.org>>, mapreduce-...@hadoop.apache.org<mailto:mapreduce-...@hadoop.apache.org> Cc: "vino...@apache.org<mailto:vino...@apache.org>" <vino...@apache.org<mailto:vino...@apache.org>> Agreed on not mixing this with major release discussions. Okay, I just finished my review of 2.8 content. A quick summary follows. Current state of originally planned items - Nearly Done / Done and so need to close down quickly — Support *both* JDK7 and JDK8 runtimes HADOOP-11090 — Supporting non-exclusive node-labels: YARN-3214: Done, can push as an alpha / beta feature — Support priorities across applications within the same queue YARN-1963: Can push as an alpha / beta feature - Definitely have to move out into 2.9 and beyond — Early work for disk and network isolation in YARN: YARN-2139, YARN-2140: Early noise, some critical pieces designed, done but not a lot of movement of late — Classpath isolation for downstream clients HADOOP-11656: Lots of chatter a while ago, not much movement of late — Support for Erasure Codes in HDFS HDFS-7285<https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the interest of stability / bake-in Non-planned features that went into 2.8.0 — The overall list of new features: https://issues.apache.org/jira/issues/?filter=12333994 — HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement - no dimension of alpha/betaness here. — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature? — Stability improvements ready to use: — HDFS-8008 Support client-side back off when the datanodes are congested — HDFS-8009 Signal congestion on the DataNode — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most of it anyways. Can push as an alpha feature. — YARN-1197 Support changing resources of an allocated container: Can push as an alpha/beta feature Items in progress to think about in 2 weeks — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before YARN-2928 comes around. I think this should go in given the tremendous activity recently. — YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but clearly a work in progress. Two options here — If it is safe to ship it into 2.8 in a disable manner, we can get the early code into trunk and all the way int o2.8. — If it is not safe, it organically rolls over into 2.9 — Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools. This is my plan of action for now in terms of the release itself - Cut a branch about two weeks from now - Do an RC mid next month (leaving ~4weeks since branch-cut) - As with 2.7.x series, the first release will still be called as early / alpha release in the interest of — gaining downstream adoption — wider testing, — yet reserving our right to fix any inadvertent incompatibilities introduced. If we can get answers on “Items to think about now” during this and next week, we will overall be in good shape. Thoughts? Thanks +Vinod PS:As you may have noted above, this time around, I want to do something that we’ve always wanted to do, but never explicitly did. I’m calling out readiness of each feature as they stand today so we can inform our users better of what they can start relying on in production clusters. On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmcc...@apache.org<mailto:cmcc...@apache.org>> wrote: I think it makes sense to have a 2.8 release since there are a tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x release seems like something we should consider separately since it would not have the same compatibility guarantees as a 2.8 release. There's a pretty big delta between trunk and 2.8 as well. cheers, Colin On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdoug...@apache.org<mailto:cdoug...@apache.org>> wrote: With two active sustaining branches (2.6, 2.7), what would you think of releasing trunk as 3.x instead of pushing 2.8? There are many new features (EC, Y1197, etc.), and trunk could be the source of several alpha/beta releases before we fork the 3.x line. -C On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli <vino...@hortonworks.com<mailto:vino...@hortonworks.com>> wrote: As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the unusually long 2.6.1 release. With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8. I’ll do a quick survey of where the individual features are and the amount of content already present in 2.8 and kick-start 2.8.0 process again. +Vinod On Apr 21, 2015, at 2:39 PM, vino...@apache.org<mailto:vino...@apache.org> wrote: With 2.7.0 out of the way, and with more maintenance releases to stabilize it, I propose we start thinking about 2.8.0. Here's my first cut of the proposal, will update the Roadmap wiki. - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090 - Compatibility tools to catch backwards, forwards compatibility issues at patch submission, release times. Some of it is captured at YARN-3292. This also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools. - HADOOP-11656 Classpath isolation for downstream clients - Support for Erasure Codes in HDFS HDFS-7285 - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140 - YARN Timeline Service Next generation: YARN-2928. At least branch-merge + early peek. - Supporting non-exclusive node-labels: YARN-3214 I'm experimenting with more agile 2.7.x releases and would like to continue the same by volunteering as the RM for 2.8.x too. Given the long time we took with 2.7.0, the timeline I am looking at is 8-12 weeks. We can pick as many features as they finish along and make a more predictable releases instead of holding up releases for ever. Thoughts? Thanks +Vinod
