Thanks to Vinod for starting this discussion! +1 to add YARN-1197 (container resizing) to 2.8.0, it is end-to-end tested. I'd prefer to push it as an Alpha feature before wilder testing.
And also agree to call first release of 2.8 as an Alpha release according to the number of new features / code changes. Regards, Wangda On Wed, Nov 11, 2015 at 12:13 PM, Vinod Vavilapalli <vino...@hortonworks.com > wrote: > Agreed on not mixing this with major release discussions. > > Okay, I just finished my review of 2.8 content. > > A quick summary follows. > > Current state of originally planned items > > - Nearly Done / Done and so need to close down quickly > — Support *both* JDK7 and JDK8 runtimes HADOOP-11090 > — Supporting non-exclusive node-labels: YARN-3214: Done, can push as > an alpha / beta feature > — Support priorities across applications within the same queue > YARN-1963: Can push as an alpha / beta feature > > - Definitely have to move out into 2.9 and beyond > — Early work for disk and network isolation in YARN: YARN-2139, > YARN-2140: Early noise, some critical pieces designed, done but not a lot > of movement of late > — Classpath isolation for downstream clients HADOOP-11656: Lots of > chatter a while ago, not much movement of late > — Support for Erasure Codes in HDFS HDFS-7285< > https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in the > interest of stability / bake-in > > Non-planned features that went into 2.8.0 > > — The overall list of new features: > https://issues.apache.org/jira/issues/?filter=12333994 > — HDFS-6200 Create a separate jar for hdfs-client: Compatible > improvement - no dimension of alpha/betaness here. > — HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature? > — Stability improvements ready to use: > — HDFS-8008 Support client-side back off when the datanodes are > congested > — HDFS-8009 Signal congestion on the DataNode > — YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well > most of it anyways. Can push as an alpha feature. > — YARN-1197 Support changing resources of an allocated container: Can > push as an alpha/beta feature > > Items in progress to think about in 2 weeks > > — YARN Timeline Service v1.5 - YARN-4233: A short term bridge before > YARN-2928 comes around. I think this should go in given the tremendous > activity recently. > — YARN Timeline Service Next generation: YARN-2928: Lots of momentum, > but clearly a work in progress. Two options here > — If it is safe to ship it into 2.8 in a disable manner, we can > get the early code into trunk and all the way int o2.8. > — If it is not safe, it organically rolls over into 2.9 > — Compatibility tools to catch backwards, forwards compatibility > issues at patch submission, release times. Some of it is captured at > YARN-3292. This also involves resurrecting jdiff > (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or investing in new tools. > > This is my plan of action for now in terms of the release itself > > - Cut a branch about two weeks from now > - Do an RC mid next month (leaving ~4weeks since branch-cut) > - As with 2.7.x series, the first release will still be called as early / > alpha release in the interest of > — gaining downstream adoption > — wider testing, > — yet reserving our right to fix any inadvertent incompatibilities > introduced. > > If we can get answers on “Items to think about now” during this and next > week, we will overall be in good shape. > > Thoughts? > > Thanks > +Vinod > PS:As you may have noted above, this time around, I want to do something > that we’ve always wanted to do, but never explicitly did. I’m calling out > readiness of each feature as they stand today so we can inform our users > better of what they can start relying on in production clusters. > > > On Oct 5, 2015, at 11:53 AM, Colin P. McCabe <cmcc...@apache.org<mailto: > cmcc...@apache.org>> wrote: > > I think it makes sense to have a 2.8 release since there are a > tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x > release seems like something we should consider separately since it > would not have the same compatibility guarantees as a 2.8 release. > There's a pretty big delta between trunk and 2.8 as well. > > cheers, > Colin > > On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas <cdoug...@apache.org > <mailto:cdoug...@apache.org>> wrote: > With two active sustaining branches (2.6, 2.7), what would you think > of releasing trunk as 3.x instead of pushing 2.8? There are many new > features (EC, Y1197, etc.), and trunk could be the source of several > alpha/beta releases before we fork the 3.x line. -C > > On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli > <vino...@hortonworks.com<mailto:vino...@hortonworks.com>> wrote: > As you may have noted, 2.8.0 got completely derailed what with 2.7.x and > the unusually long 2.6.1 release. > > With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 > and 2.7.2, it’s time for us to look back at where we are with Hadoop 2.8. > > I’ll do a quick survey of where the individual features are and the amount > of content already present in 2.8 and kick-start 2.8.0 process again. > > +Vinod > > > On Apr 21, 2015, at 2:39 PM, vino...@apache.org<mailto:vino...@apache.org> > wrote: > > With 2.7.0 out of the way, and with more maintenance releases to stabilize > it, I propose we start thinking about 2.8.0. > > Here's my first cut of the proposal, will update the Roadmap wiki. > - Support *both* JDK7 and JDK8 runtimes: HADOOP-11090 > - Compatibility tools to catch backwards, forwards compatibility issues at > patch submission, release times. Some of it is captured at YARN-3292. This > also involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) > and/or investing in new tools. > - HADOOP-11656 Classpath isolation for downstream clients > - Support for Erasure Codes in HDFS HDFS-7285 > - Early work for disk and network isolation in YARN: YARN-2139, YARN-2140 > - YARN Timeline Service Next generation: YARN-2928. At least branch-merge > + early peek. > - Supporting non-exclusive node-labels: YARN-3214 > > I'm experimenting with more agile 2.7.x releases and would like to > continue the same by volunteering as the RM for 2.8.x too. > > Given the long time we took with 2.7.0, the timeline I am looking at is > 8-12 weeks. We can pick as many features as they finish along and make a > more predictable releases instead of holding up releases for ever. > > Thoughts? > > Thanks > +Vinod > > > >
