I added my comments on that JIRA. Looks like YARN-7292 is marked as a
blocker for 3.1, and I would tend to agree with that. Let's see what we
can to do get profiles nailed down so that 3.1 can go forward.
Daniel
On 1/18/18 10:25 AM, Wangda Tan wrote:
Thanks Daniel,
We need to make a decision about this:
https://issues.apache.org/jira/browse/YARN-7292, I believe this is the
JIRA you mentioned correct? Please let me know if there's anything
else. And let's move the discussion on JIRA.
The good news is that resource profile is merged to trunk already so
we can finish that before code freeze date (Feb 08).
+ Sunil as well.
Thanks,
Wangda
On Wed, Jan 17, 2018 at 4:31 PM, Daniel Templeton <dan...@cloudera.com
<mailto:dan...@cloudera.com>> wrote:
What's the status on resource profiles? I believe there are still
a couple of open JIRAs to rethink some of the design choices.
Daniel
On 1/17/18 11:33 AM, Wangda Tan wrote:
Hi All,
Since we're fast approaching previously proposed feature
freeze date (Jan
30, about 13 days from today). If you've any features which
live in a
branch and targeted to 3.1.0, please reply this email thread.
Ideally, we
should finish branch merging before feature freeze date.
Here's an updated 3.1.0 feature status:
1. Merged & Completed features:
* (Sunil) YARN-5881: Support absolute value in CapacityScheduler.
* (Wangda) YARN-6223: GPU support on YARN. Features in trunk
and works
end-to-end.
* (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN native
services.
* (Steve Loughran): HADOOP-13786: S3Guard committer for
zero-rename commits.
* (Suma): YARN-7117: Capacity Scheduler: Support Auto Creation
of Leaf
Queues While Doing Queue Mapping.
* (Chris Douglas) HDFS-9806: HDFS Tiered Storage.
2. Features close to finish:
* (Zhankun) YARN-5983: FPGA support. Majority implementations
completed and
merged to trunk. Except for UI/documentation.
* (Uma) HDFS-10285: HDFS SPS. Majority implementations are
done, some
discussions going on about implementation.
* (Arun Suresh / Kostas / Wangda). YARN-6592: New
SchedulingRequest and
anti-affinity support. Close to finish, on track to be merged
before Jan 30.
3. Tentative features:
* (Arun Suresh). YARN-5972: Support pausing/freezing opportunistic
containers. Only one pending patch. Plan to finish before Jan 7th.
* (Haibo Chen). YARN-1011: Resource overcommitment. Looks
challenging to be
done before Jan 2018.
* (Anu): HDFS-7240: Ozone. Given the discussion on HDFS-7240.
Looks
challenging to be done before Jan 2018.
* (Varun V) YARN-5673: container-executor write. Given
security refactoring
of c-e (YARN-6623) is already landed, IMHO other stuff may be
moved to 3.2.
Thanks,
Wangda
On Fri, Dec 15, 2017 at 1:20 PM, Wangda Tan
<wheele...@gmail.com <mailto:wheele...@gmail.com>> wrote:
Hi all,
Congratulations on the 3.0.0-GA release!
As we discussed in the previous email thread [1], I'd like
to restart
3.1.0 release plans.
a) Quick summary:
a.1 Release status
We started 3.1 release discussion on Sep 6, 2017 [1]. As
of today,
there’re 232 patches loaded on 3.1.0 alone [2], besides 6
open blockers and
22 open critical issues.
a.2 Release date update
Considering delays of 3.0-GA release by month-and-a-half,
I propose to
move the dates as follows
- feature freeze date from Dec 15, 2017, to Jan 30, 2018
- last date for
any branches to get merged too;
- code freeze (blockers & critical only) date to Feb 08,
2018;
- release voting start by Feb 18, 2018, leaving time for
at least two RCx
- release date from Jan 15, 2018, to Feb 28, 2018;
Unlike before, I added an additional milestone for
release-vote-start so
that we can account for voting time-period also.
This overall is still 5 1/2 months of release-timeline
unlike the faster
cadence we hoped for, but this, in my opinion, is the
best-updated timeline
given the delays of the final release of 3.0-GA.
b) Individual feature status:
I spoke to several feature owners and checked the status
of un-finished
features, following are status of features planned to 3.1.0:
b.1 Merged & Completed features:
* (Sunil) YARN-5881: Support absolute value in
CapacityScheduler.
* (Wangda) YARN-6223: GPU support on YARN. Features in
trunk and works
end-to-end.
* (Jian) YARN-5079,YARN-4793,YARN-4757,YARN-6419 YARN
native services.
* (Steve Loughran): HADOOP-13786: S3Guard committer for
zero-rename
commits.
* (Suma): YARN-7117: Capacity Scheduler: Support Auto
Creation of Leaf
Queues While Doing Queue Mapping.
b.2 Features close to finish:
* (Chris Douglas) HDFS-9806: HDFS Tiered Storage. Being
voting now.
* (Zhankun) YARN-5983: FPGA support. Majority
implementations completed
and merged to trunk. Except for UI/documentation.
* (Uma) HDFS-10285: HDFS SPS. Majority implementations are
done, some
discussions going on about implementation.
b.3 Tentative features:
* (Arun Suresh). YARN-5972: Support pausing/freezing
opportunistic
containers. Only one pending patch. Plan to finish before
Jan 7th.
* (Haibo Chen). YARN-1011: Resource overcommitment. Looks
challenging to
be done before Jan 2018.
* (Arun Suresh / Kostas / Wangda). YARN-6592: New
SchedulingRequest and
anti-affinity support. Tentative will figure out by Jan 1st.
* (Anu): HDFS-7240: Ozone. Given the discussion on
HDFS-7240. Looks
challenging to be done before Jan 2018.
* (Varun V) YARN-5673: container-executor write. Given
security
refactoring of c-e (YARN-6623) is already landed, IMHO
other stuff may be
moved to 3.2.
b.4 Additional release drivers
* More exhaustive upgrade testing from 2.x to 3.x.
c) Regarding branch cut:
We will keep pointing trunk to 3.1 and cut branch-3.1
until: A. some
feature planned to 3.2 has to be landed on trunk or B.
After feature freeze
date, whichever comes first.
I've also talked offline with Vinod to get help on
release-management
given this is my first release. He agreed to help do this
release jointly.
Thoughts?
Thanks,
Wangda Tan
[1]
https://lists.apache.org/thread.html/c11506c3250c9481852130616b3cb0
<https://lists.apache.org/thread.html/c11506c3250c9481852130616b3cb0>
9a0e222f5c2465c015f9906dab@%3Cyarn-dev.hadoop.apache.org
<http://3Cyarn-dev.hadoop.apache.org>%3E
[2] "project in (YARN, HADOOP, MAPREDUCE, HDFS) AND
fixVersion in (3.1.0)
AND fixVersion not in (3.0.0,2.9.0) ORDER BY priority DESC”
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
<mailto:common-dev-unsubscr...@hadoop.apache.org>
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
<mailto:common-dev-h...@hadoop.apache.org>
