Hi Vinod and everyone else who helped on this release!
Thank you very much for going through the work and effort to put this release
together.
While doing my testing for Hadoop 2.6.1 RC1, I encountered the following
issue:[MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon
IOException during InMemoryMapOutput shuffle handler - ASF JIRA
| |
| | | | | |
| [MAPREDUCE-6334] Fetcher#copyMapOutput is leaking usedMemory upon IOException
during ...We are seeing this happen when an NM's disk goes bad during the
creation of map output(s) the reducer's fetcher can read the shuffle header and
reserve the memory |
| |
| View on issues.apache.org | Preview by Yahoo |
| |
| |
This is actually something we ran across frequently when running 2.6. You may
want to consider pulling that in to 2.6.
Other than that, it looks fine. I did the following manual testing on a
one-node cluster:
- successfully downloaded and compiled the code- successfully ran streaming
jobs- successfully ran distributed shell jobs-- as part of the distributed
shell testing, I started the jobs with the
'-keep_containers_across_application_attempts' property set, killed the AM
container, and verified that it would in fact keep the containers running
across AM restarts.- sucessfully ran wordcount jobs.
Thank you,
-Eric Payne
From: Vinod Kumar Vavilapalli <vino...@apache.org>
To: common-dev@hadoop.apache.org; yarn-...@hadoop.apache.org;
hdfs-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org
Cc: vino...@apache.org; Sangjin Lee <sjl...@gmail.com>; Akira AJISAKA
<ajisa...@oss.nttdata.co.jp>
Sent: Wednesday, September 16, 2015 9:10 PM
Subject: [VOTE] Release Apache Hadoop 2.6.1 RC1
Hi all,
After a nearly month long [1] toil, with loads of help from Sangjin Lee and
Akira Ajisaka, and 153 (RC0)+7(RC1) commits later, I've created a release
candidate RC1 for hadoop-2.6.1.
RC1 is RC0 [0] (for which I opened and closed a vote last week) + UI fixes
for the issue Sangjin raised (YARN-3171 and the dependencies YARN-3779,
YARN-3248), additional fix to avoid incompatibility (YARN-3740), other UI
bugs (YARN-1884, YARN-3544) and the MiniYARNCluster issue (right patch for
YARN-2890) that Jeff Zhang raised.
The RC is available at: http://people.apache.org/~vinodkv/hadoop-2.6.1-RC1/
The RC tag in git is: release-2.6.1-RC1
The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1021
Some notes from our release process
- - Sangjin and I moved out a bunch of items pending from 2.6.1 [2] -
non-committed but desired patches. 2.6.1 is already big as is and is late
by any standard, we can definitely include them in the next release.
- The 2.6.1 wiki page [3] captures some (but not all) of the context of
the patches that we pushed in.
- Given the number of fixes pushed [4] in, we had to make a bunch of
changes to our original plan - we added a few improvements that helped us
backport patches easier (or in many cases made backports possible), and we
dropped a few that didn't make sense (HDFS-7831, HDFS-7926, HDFS-7676,
HDFS-7611, HDFS-7843, HDFS-8850).
- I ran all the unit tests which (surprisingly?) passed. (Except for one,
which pointed out a missing fix HDFS-7552).
As discussed before [5]
- This release is the first point release after 2.6.0
- I’d like to use this as a starting release for 2.6.2 in a few weeks and
then follow up with more of these.
Please try the release and vote; the vote will run for the usual 5 days.
Thanks,
Vinod
[0] Hadoop 2.6.1 RC0 vote: http://markmail.org/thread/ubut2rn3lodc55iy
[1] Hadoop 2.6.1 Release process thread:
http://markmail.org/thread/wkbgkxkhntx5tlux
[2] 2.6.1 Pending tickets:
https://issues.apache.org/jira/issues/?filter=12331711
[3] 2.6.1 Wiki page: https://wiki.apache.org/hadoop/Release-2.6.1
-Working-Notes
[4] List of 2.6.1 patches pushed:
https://issues.apache.org/jira/issues/?jql=fixVersion%20%3D%202.6.1
%20and%20labels%20%3D%20%222.6.1-candidate%22
[5] Planning Hadoop 2.6.1 release:
http://markmail.org/thread/sbykjn5xgnksh6wg
PS:
- Note that branch-2.6 which will be the base for 2.6.2 doesn't have these
fixes yet. Once 2.6.1 goes through, I plan to rebase branch-2.6 based off
2.6.1.
- The additional patches in RC1 that got into 2.6.1 all the way from 2.8
are NOT in 2.7.2 yet, this will be done as a followup.
