Re: Logistics for releasing 2.4

2014-01-24 Thread Kihwal Lee
Thanks for the list, Jason.  I scanned through them and except HDFS-5356, all jiras were currently targeted for 2.4.0. I changed the target version of HDFS-5356 to 2.4.0, so now all are targeted for 2.4.0. Assuming "2.3.0" label is clean and doesn't need any mass/batch update, I think we can go

Re: Logistics for releasing 2.4

2014-01-24 Thread Jason Lowe
I just committed the addendum patch for HADOOP-9652 which should resolve the performance issue. Speaking of critical issues to fix for 2.3, I do wonder what to do with all the Blocker/Criticals targeted for 2.4 (the old branch-2 HEAD release) which apparently is now going to be 2.3 (the new br

Re: Logistics for releasing 2.4

2014-01-23 Thread Sangjin Lee
I also wanted to bring your attention to HADOOP-9652. As this would address a major performance issue with the local filesystem, it would be good to get this in for 2.4 (or is it 2.3?). Regards, Sangjin On Thu, Jan 23, 2014 at 2:23 AM, Steve Loughran wrote: > the JIRA I referenced is ready to g

Re: Logistics for releasing 2.4

2014-01-23 Thread Steve Loughran
the JIRA I referenced is ready to go -it just needs review On 22 January 2014 20:04, Andrew Wang wrote: > Thanks for the comments everyone. > > Vinod, if you think YARN-149 isn't ready yet, we can leave it out. > Alternatively, we could release note it as "beta" with said known issues, > and le

Re: Logistics for releasing 2.4

2014-01-23 Thread Vinod Kumar Vavilapalli
Can you please send out a separate note so that others who aren't following this thread also know about the change of direction? And update the RoadMap wiki too? +Vinod On Jan 22, 2014, at 6:01 PM, Arun Murthy wrote: > Thanks Vinod. > > I'll start the work to re-spin 2.3 by changing jira n

Re: Logistics for releasing 2.4

2014-01-22 Thread Stack
On Wed, Jan 22, 2014 at 5:21 PM, Arun C Murthy wrote: > Andrew, > > > On Jan 22, 2014, at 12:04 PM, Andrew Wang > wrote: > > > Monthly releases make no sense to me. > > I think the 'monthly' was a rough measure. Every 6-8 weeks works too. A regular release schedule works nicely over in hbasela

Re: Logistics for releasing 2.4

2014-01-22 Thread Arpit Gupta
There are a couple of minor doc changes if they can be included in the next release that will be great. https://issues.apache.org/jira/browse/HADOOP-10050 https://issues.apache.org/jira/browse/HADOOP-10065 Thanks -- Arpit Gupta Hortonworks Inc. http://hortonworks.com/ On Jan 22, 2014, at 6:09

Re: Logistics for releasing 2.4

2014-01-22 Thread Arun Murthy
Sounds good, thanks. I'll ping you off-thread. Arun On Wed, Jan 22, 2014 at 6:01 PM, Andrew Wang wrote: > Hi Arun, thanks for the reply. I hope that I've come across as an earnest > and motivated contributor who wants to help make high-quality releases. > Volunteering myself as RM and pushing f

Re: Logistics for releasing 2.4

2014-01-22 Thread Andrew Wang
Hi Arun, thanks for the reply. I hope that I've come across as an earnest and motivated contributor who wants to help make high-quality releases. Volunteering myself as RM and pushing for a 2.4 was not a reflection on your stewardship of 2.x so far. I know it's a lot of work. I didn't mean to sugg

Re: Logistics for releasing 2.4

2014-01-22 Thread Arun Murthy
Thanks Vinod. I'll start the work to re-spin 2.3 by changing jira numbers, CHANGES.txt etc. now. Arun On Wed, Jan 22, 2014 at 5:59 PM, Vinod Kumar Vavilapalli wrote: > > On Jan 22, 2014, at 5:36 PM, Arun C Murthy wrote: > > Agree 2.3 is now very stale. > > So, as I understand, your motivatio

Re: Logistics for releasing 2.4

2014-01-22 Thread Vinod Kumar Vavilapalli
On Jan 22, 2014, at 5:36 PM, Arun C Murthy wrote: > Agree 2.3 is now very stale. > > So, as I understand, your motivation is to get HDFS-2832 & HDFS-4949 out, > correct? > > If so, that could be a better option - I can abandon the 2.3rc, and then > release current branch-2 as 2.3. Would that

Re: Logistics for releasing 2.4

2014-01-22 Thread Vinod Kumar Vavilapalli
> As to a more frequent release cadence, it seems like so far there's a lot > of support. I think downstreams would actually appreciate more frequent > releases. HBase at least only builds against release artifacts, so they end > up scrambling whenever we drop a new release. Releasing more often w

Re: Logistics for releasing 2.4

2014-01-22 Thread Arun C Murthy
On Jan 22, 2014, at 5:14 PM, Andrew Wang wrote: > Another option (not to step on anyone's toes) is to scratch the current 2.3 > branch and make HDFS-2832+HDFS-4949 the new 2.3, since the current 2.3 has > sort of lost its way as a bugfix-only release, and has diverged quite a bit > from branch-2

Re: Logistics for releasing 2.4

2014-01-22 Thread Arun C Murthy
Andrew, > On Jan 22, 2014, at 12:04 PM, Andrew Wang wrote: Monthly releases make no sense to me. They would result in a number of meaningless releases (of the 12 releases per year) of which most would be unstable and would lack features most folks want (for e.g. Yahoo has been clear that Rol

Re: Logistics for releasing 2.4

2014-01-22 Thread Andrew Wang
Hi Vinod, If the main point of contention is the alpha/beta/stable labeling for features, we can just not include anything but stable features in the proposed 2.4. This means HDFS-2832 and HDFS-4949 (along with the plethora of other improvements that have gone in since 2.2). AFAIK the previous pla

Re: Logistics for releasing 2.4

2014-01-22 Thread Vinod Kumar Vavilapalli
I think YARN-149 will be ready if we do a 2.4 release sometime next month. The same goes for YARN-321 also. The merge-to-trunk vote is happening now and it should be in a good shape soon if we stick to the early plan that is laid out in the roadmap. Overall, a bunch of us in the YARN side have

Re: Logistics for releasing 2.4

2014-01-22 Thread Karthik Kambatla
+1 to releasing often. Also, like Vinod's idea of labeling individual features alpha/beta/stable. I am +1 to releasing 2.4 end of this month and including YARN-149 as "beta". I think the associated configs are stable; we might add new ones to make it simpler. We have been testing it internally in

Re: Logistics for releasing 2.4

2014-01-22 Thread Andrew Wang
Thanks for the comments everyone. Vinod, if you think YARN-149 isn't ready yet, we can leave it out. Alternatively, we could release note it as "beta" with said known issues, and let people kick the tires. It looks like a bunch of the core functionality is already in place. Unless anyone else obj

Re: Logistics for releasing 2.4

2014-01-22 Thread Stack
Thanks Andrew for bringing this up. +1 on more frequent releases and an effort at (roughly time-based release. We are working to get 'HDFS-5776 Support 'hedged' reads in DFSClient' to land in time for 2.4 (but don't hold up the release for us!) St.Ack On Tue, Jan 21, 2014 at 3:51 PM, Vinod Kum

Re: Logistics for releasing 2.4

2014-01-21 Thread Vinod Kumar Vavilapalli
If the timeline is to cut one next week, I don't think we can ship YARN-149 as part of that and call it stable. There are a bunch of major things that are still missing there: YARN-1202, YARN-1410, YARN-1525 and YARN-1611/YARN-1459. We need to start labeling individual features alpha/beta/stable

Re: Logistics for releasing 2.4

2014-01-21 Thread Steve Loughran
I'd like the (trivial) bug fix of HADOOP-10085 in. Patch is one line of production code (sync clone of a list in the getter), a few hundred lines of test. On 21 January 2014 21:26, Andrew Wang wrote: > Arun, great to hear 2.3 is coming down the pipe! I also agree with Suresh > that we should mo

Re: Logistics for releasing 2.4

2014-01-21 Thread Andrew Wang
Arun, great to hear 2.3 is coming down the pipe! I also agree with Suresh that we should move ahead without symlinks, which is why I moved it out of the 2.4 section on the roadmap wiki page. In terms of other 2.4 features, it seems risky to try and squeeze this many new features into branch-2 in t

Re: Logistics for releasing 2.4

2014-01-21 Thread Sandy Ryza
+1 for a 2.4 (and 2.3) this month. The current vote for YARN-321 is to merge it to trunk. Rereading the discussion around YARN-321, my understanding is that it's unlikely to be stable it by the end of the month. It still needs security and API stabilization work. If we end up in a situation whe

Re: Logistics for releasing 2.4

2014-01-21 Thread Suresh Srinivas
There is not much progress on symlinks issue. I think we should move forward with 2.4 release with symlinks disabled. Status of 2.4 features from HDFS so far: - HDFS-2832 Heterogeneous storage support has been merged - HDFS-5535 rolling upgrades work is in progress - HDFS-4685 ACL related work is

Re: Logistics for releasing 2.4

2014-01-21 Thread Arun C Murthy
Andrew, I'm almost ready to push out rc0 for 2.3 (been testing it overnight), I'm pretty sure I'll get that out tonight. However, AHS (YARN-321) is very close (merge vote going on) … so that will definitely make it in very soon. So, my plan is essentially the same i.e. release 2.4 end of th

Logistics for releasing 2.4

2014-01-20 Thread Andrew Wang
Hi all, I'm pretty excited to see a 2.4 this month if possible. Since I think people were favorable to the idea of time-based releases, how do we feel about just cutting branch-2 and spinning up the release process for our January goal? Looking at the roadmap (https://wiki.apache.org/hadoop/Roadm