Agreed on not mixing this with major release discussions.
Okay, I just finished my review of 2.8 content.
A quick summary follows.
Current state of originally planned items
- Nearly Done / Done and so need to close down quickly
— Support *both* JDK7 and JDK8 runtimes HADOOP-11090
— Supporting non-exclusive node-labels: YARN-3214: Done, can push as an
alpha / beta feature
— Support priorities across applications within the same queue YARN-1963:
Can push as an alpha / beta feature
- Definitely have to move out into 2.9 and beyond
— Early work for disk and network isolation in YARN: YARN-2139, YARN-2140:
Early noise, some critical pieces designed, done but not a lot of movement of
late
— Classpath isolation for downstream clients HADOOP-11656: Lots of chatter
a while ago, not much movement of late
— Support for Erasure Codes in HDFS
HDFS-7285<https://issues.apache.org/jira/browse/HDFS-7285>: Moved out to 2.9 in
the interest of stability / bake-in
Non-planned features that went into 2.8.0
— The overall list of new features:
https://issues.apache.org/jira/issues/?filter=12333994
— HDFS-6200 Create a separate jar for hdfs-client: Compatible improvement -
no dimension of alpha/betaness here.
— HDFS-8155 Support OAuth2 in WebHDFS: Alpha / Early feature?
— Stability improvements ready to use:
— HDFS-8008 Support client-side back off when the datanodes are
congested
— HDFS-8009 Signal congestion on the DataNode
— YARN-3611 Support Docker Containers In LinuxContainerExecutor: Well most
of it anyways. Can push as an alpha feature.
— YARN-1197 Support changing resources of an allocated container: Can push
as an alpha/beta feature
Items in progress to think about in 2 weeks
— YARN Timeline Service v1.5 - YARN-4233: A short term bridge before
YARN-2928 comes around. I think this should go in given the tremendous activity
recently.
— YARN Timeline Service Next generation: YARN-2928: Lots of momentum, but
clearly a work in progress. Two options here
— If it is safe to ship it into 2.8 in a disable manner, we can get the
early code into trunk and all the way int o2.8.
— If it is not safe, it organically rolls over into 2.9
— Compatibility tools to catch backwards, forwards compatibility issues at
patch submission, release times. Some of it is captured at YARN-3292. This also
involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or
investing in new tools.
This is my plan of action for now in terms of the release itself
- Cut a branch about two weeks from now
- Do an RC mid next month (leaving ~4weeks since branch-cut)
- As with 2.7.x series, the first release will still be called as early /
alpha release in the interest of
— gaining downstream adoption
— wider testing,
— yet reserving our right to fix any inadvertent incompatibilities
introduced.
If we can get answers on “Items to think about now” during this and next week,
we will overall be in good shape.
Thoughts?
Thanks
+Vinod
PS:As you may have noted above, this time around, I want to do something that
we’ve always wanted to do, but never explicitly did. I’m calling out readiness
of each feature as they stand today so we can inform our users better of what
they can start relying on in production clusters.
On Oct 5, 2015, at 11:53 AM, Colin P. McCabe
<cmcc...@apache.org<mailto:cmcc...@apache.org>> wrote:
I think it makes sense to have a 2.8 release since there are a
tremendous number of JIRAs in 2.8 that are not in 2.7. Doing a 3.x
release seems like something we should consider separately since it
would not have the same compatibility guarantees as a 2.8 release.
There's a pretty big delta between trunk and 2.8 as well.
cheers,
Colin
On Sat, Sep 26, 2015 at 1:36 PM, Chris Douglas
<cdoug...@apache.org<mailto:cdoug...@apache.org>> wrote:
With two active sustaining branches (2.6, 2.7), what would you think
of releasing trunk as 3.x instead of pushing 2.8? There are many new
features (EC, Y1197, etc.), and trunk could be the source of several
alpha/beta releases before we fork the 3.x line. -C
On Sat, Sep 26, 2015 at 12:49 PM, Vinod Vavilapalli
<vino...@hortonworks.com<mailto:vino...@hortonworks.com>> wrote:
As you may have noted, 2.8.0 got completely derailed what with 2.7.x and the
unusually long 2.6.1 release.
With 2.6.1 out of the way, and two parallel threads in progress for 2.6.2 and
2.7.2, it’s time for us to look back at where we are with Hadoop 2.8.
I’ll do a quick survey of where the individual features are and the amount of
content already present in 2.8 and kick-start 2.8.0 process again.
+Vinod
On Apr 21, 2015, at 2:39 PM, vino...@apache.org<mailto:vino...@apache.org>
wrote:
With 2.7.0 out of the way, and with more maintenance releases to stabilize it,
I propose we start thinking about 2.8.0.
Here's my first cut of the proposal, will update the Roadmap wiki.
- Support *both* JDK7 and JDK8 runtimes: HADOOP-11090
- Compatibility tools to catch backwards, forwards compatibility issues at
patch submission, release times. Some of it is captured at YARN-3292. This also
involves resurrecting jdiff (HADOOP-11776/YARN-3426/MAPREDUCE-6310) and/or
investing in new tools.
- HADOOP-11656 Classpath isolation for downstream clients
- Support for Erasure Codes in HDFS HDFS-7285
- Early work for disk and network isolation in YARN: YARN-2139, YARN-2140
- YARN Timeline Service Next generation: YARN-2928. At least branch-merge +
early peek.
- Supporting non-exclusive node-labels: YARN-3214
I'm experimenting with more agile 2.7.x releases and would like to continue the
same by volunteering as the RM for 2.8.x too.
Given the long time we took with 2.7.0, the timeline I am looking at is 8-12
weeks. We can pick as many features as they finish along and make a more
predictable releases instead of holding up releases for ever.
Thoughts?
Thanks
+Vinod
