I just committed the addendum patch for HADOOP-9652 which should resolve
the performance issue.
Speaking of critical issues to fix for 2.3, I do wonder what to do with
all the Blocker/Criticals targeted for 2.4 (the old branch-2 HEAD
release) which apparently is now going to be 2.3 (the new branch-2 HEAD
release). We should either get these resolved before the 2.3 release or
decide which ones aren't really blockers or that critical for 2.3 and
target them for a future release (e.g.: 2.4). According to my
calculations, there's currently 24 JIRAs that need to be scrubbed
accordingly.
https://issues.apache.org/jira/issues/?jql=project%20in%20%28hadoop%2C%20hdfs%2C%20mapreduce%2C%20yarn%29%20and%20resolution%20%3D%20Unresolved%20and%20priority%20in%20%28Blocker%2C%20Critical%29%20and%20%22Target%20Version%2Fs%22%20in%20%28%222.3.0%22%2C%20%222.4.0%22%29
Jason
On 01/23/2014 06:53 PM, Sangjin Lee wrote:
I also wanted to bring your attention to HADOOP-9652. As this would address
a major performance issue with the local filesystem, it would be good to
get this in for 2.4 (or is it 2.3?).
Regards,
Sangjin
On Thu, Jan 23, 2014 at 2:23 AM, Steve Loughran <ste...@hortonworks.com>wrote:
the JIRA I referenced is ready to go -it just needs review
On 22 January 2014 20:04, Andrew Wang <andrew.w...@cloudera.com> wrote:
Thanks for the comments everyone.
Vinod, if you think YARN-149 isn't ready yet, we can leave it out.
Alternatively, we could release note it as "beta" with said known issues,
and let people kick the tires. It looks like a bunch of the core
functionality is already in place.
Unless anyone else objects, I plan to cut a 2.4 branch later this week.
Steve, Stack, if you think those two JIRAs can wrap up in that time frame
we can include it. Since this more experiment will hopefully go well, we
can include them in a 2.5 next month.
Best,
Andrew
On Wed, Jan 22, 2014 at 11:28 AM, Stack <st...@duboce.net> wrote:
Thanks Andrew for bringing this up.
+1 on more frequent releases and an effort at (roughly time-based
release.
We are working to get 'HDFS-5776 Support 'hedged' reads in DFSClient'
to
land in time for 2.4 (but don't hold up the release for us!)
St.Ack
On Tue, Jan 21, 2014 at 3:51 PM, Vinod Kumar Vavilapalli <
vino...@hortonworks.com> wrote:
If the timeline is to cut one next week, I don't think we can ship
YARN-149 as part of that and call it stable. There are a bunch of
major
things that are still missing there: YARN-1202, YARN-1410, YARN-1525
and
YARN-1611/YARN-1459.
We need to start labeling individual features alpha/beta/stable now
that
we have a stable 2.2 base.
Thanks
+Vinod
On Jan 21, 2014, at 1:26 PM, Andrew Wang <andrew.w...@cloudera.com>
wrote:
So, as per my earlier email, I think a 2.4 with just HDFS-4949,
HDFS-2832,
and YARN-149 would be an attractive and stable release, and is
something
we
could actually cut this week and vote on before the month is out.
The
other
stuff we can ship in Feb or March when it's gotten a chance to bake
for a
bit, and culturally speaking, the fact that it's in 2.5 rather than
2.4
shouldn't be a big deal.
Thanks,
Andrew
On Tue, Jan 21, 2014 at 11:37 AM, Suresh Srinivas <
sur...@hortonworks.com>wrote:
There is not much progress on symlinks issue. I think we should
move
forward with 2.4 release with symlinks disabled.
Status of 2.4 features from HDFS so far:
- HDFS-2832 Heterogeneous storage support has been merged
- HDFS-5535 rolling upgrades work is in progress
- HDFS-4685 ACL related work is close to completion
- HDFS-4949 As Andrew has proposed, this will be soon merged into
2.4
Regards,
Suresh
On Tue, Jan 21, 2014 at 11:12 AM, Arun C Murthy <
a...@hortonworks.com>
wrote:
Andrew,
I'm almost ready to push out rc0 for 2.3 (been testing it
overnight),
I'm
pretty sure I'll get that out tonight.
However, AHS (YARN-321) is very close (merge vote going on) … so
that
will definitely make it in very soon.
So, my plan is essentially the same i.e. release 2.4 end of the
month
(after a bit more testing of RM HA in secure mode). Thanks for
the
offer,
I'll ping you if I need any help.
OTOH, can someone from HDFS chime in on status of symlinks?
Arun
On Jan 20, 2014, at 4:19 PM, Andrew Wang <
andrew.w...@cloudera.com
wrote:
Hi all,
I'm pretty excited to see a 2.4 this month if possible. Since I
think
people were favorable to the idea of time-based releases, how do
we
feel
about just cutting branch-2 and spinning up the release process
for
our
January goal?
Looking at the roadmap (https://wiki.apache.org/hadoop/Roadmap
),
on
the
HDFS side, I plan to post a branch-2 patch for HDFS-4949 this
week,
and
HDFS-2832 is already in. On the YARN side, it appears that RM HA
is
in,
but
the other three features (AHS, unmanaged containers, and dynamic
resource
configuration) remain unresolved.
I think a 2.4 with HDFS-4949, HDFS-2832, and YARN-149 is
already a
pretty
nice release. If it'd help, I'm willing to volunteer as release
manager
to
help get this out the door.
Thoughts welcome!
Thanks,
Andrew
--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
entity
to
which it is addressed and may contain information that is
confidential,
privileged and exempt from disclosure under applicable law. If
the
reader
of this message is not the intended recipient, you are hereby
notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you
have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.
--
http://hortonworks.com/download/
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
entity to
which it is addressed and may contain information that is
confidential,
privileged and exempt from disclosure under applicable law. If the
reader
of this message is not the intended recipient, you are hereby
notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you
have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or
entity
to
which it is addressed and may contain information that is
confidential,
privileged and exempt from disclosure under applicable law. If the
reader
of this message is not the intended recipient, you are hereby
notified
that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender
immediately
and delete it from your system. Thank You.
--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
