Re: Setting JIRA fix versions for 3.0.0 releases

2016-08-25 Thread Andrew Wang
I've taken a look at the Common and HDFS compat issues. * A bunch of them are from the known removals of the deprecated Metrics 1 and RCC. JACC understands and can filter @Deprecated annotations, but HADOOP-7266 isn't in 2.7.2, and apparently JACC doesn't look at the class annotation when telling

Re: Setting JIRA fix versions for 3.0.0 releases

2016-08-25 Thread Andrew Wang
Sean gave me some pointers on using Java ACC, I've made a report using the script I've been working on at YETUS-445: http://home.apache.org/~wang/h3/report.html Invocation was something like this: -> % ~/dev/yetus/check-java-compatibility/checkjavacompatibility.py --annotation org.apache.hadoop.

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-26 Thread Wangda Tan
Hi all, I'm trying to generate JDiff for sub projects of Hadoop, some updates: *- Common*: JDiff cannot be generated , filed https://issues.apache.org/jira/browse/HADOOP-13428 and debugging that. - *HDFS*: It pointed to a older version (2.6.0), we need to upgrade it to the latest stable release (p

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-26 Thread Vinod Kumar Vavilapalli
+1 Thanks +Vinod > On Jul 26, 2016, at 7:39 AM, Wangda Tan wrote: > > lets try to use both jdiff and the new tool and compare results because this > is the first time with the new tool. > > Appreciate your time to help us about this effort!

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-26 Thread Wangda Tan
Hi Sean, Sorry I didn't make it clear, lets try to use both jdiff and the new tool and compare results because this is the first time with the new tool. Appreciate your time to help us about this effort! Thanks, Wangda Sent from my iPhone > On Jul 26, 2016, at 6:01 AM, Sean Busbey wrote: > >

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-26 Thread Sean Busbey
Just so I don't waste time chasing my tail, should I interpret this email and the associated JIRA as the PMC preferring I not spend volunteer time providing a compatibility breakdown as previously discussed? On Mon, Jul 25, 2016 at 7:54 PM, Wangda Tan wrote: > I just filed ticket https://issues.a

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-26 Thread Sean Busbey
Yes, the Java API Compliance Checker allows specifying Annotations to pare down where incompatible changes happen. It was added some time ago based on feedback from the Apache HBase project. The limitations I've found are: 1) at least earlier versions only supported annotations at the class level

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Andrew Wang
Thanks Vinod+Wangda for the comments, Above, Allen and I discussed a proposal which I think is reasonable and also lines up well with our current approach. If you feel something is underspecified or needs improvement, please raise those points. We have been doing concurrent releases with the 2.6.

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Wangda Tan
Hi Andrew, Please wait updating fix version for branch-2 committed tickets before we get a consensus on this. Updating fix versions for them could bring lots of troubles for on going two releases (2.7.3 / 2.8.0). Thanks, Wangda On Mon, Jul 25, 2016 at 11:02 AM, Andrew Wang wrote: > If I don't h

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Wangda Tan
I just filed ticket https://issues.apache.org/jira/browse/HADOOP-13423 to track running JDIFF on trunk and analyze results for Hadoop-common. I will work on that and keep the JIRA and this thread updated. We need to do the same work for YARN/MR/HDFS. On Mon, Jul 25, 2016 at 5:47 PM, Wangda Tan wr

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Wangda Tan
I agree with what Vinod mentioned: we need to revisit all incompatible changes and revert unnecessary ones. Even if we don't have any compatibility guarantees between 2.x and 3.x. But make user to be less frustrated while trying 3.x is always a better option to me. To achieve this we need to run j

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Vinod Kumar Vavilapalli
Please don’t do this for 2.8.0 against 2.7.0 till we change our existing release ordering conventions. Till now, I made sure 2.8.0 only has issues that are not in 2.7.3 - per the 2.8.0-follows-2.7.3 convention. As part of the release process, I will fix the remaining tickets to follow this same

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Vinod Kumar Vavilapalli
>> There's a plan for more 3.0.0 alphas, so there's still time to help out > before things settle down for beta. If 2.8.0 is ready to go, it should > happen before even alpha2. I wasn’t talk about my irresistible urge to help (which of course is there : )) , it was about making sure enough eye-ba

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Vinod Kumar Vavilapalli
Actually, I wouldn’t trust this report as it stands today at all. I quickly glanced the report, looking for what it highlights as incompatible. But the ones that I saw have private / visible for testing annotations. Other than acting as useless evidence for those lashing out on branch-2, this wo

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-25 Thread Andrew Wang
If I don't hear otherwise, I'd like to go ahead and do the bulk fix version update on JIRA for 3.0.0-alpha1, diffing from 2.7.0. Will wait until EOD tomorrow in case there's more discussion. If any one is willing to help review my script and JIRA queries, that'd also be great. I can also do the 2.

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Allen Wittenauer
> On Jul 22, 2016, at 7:16 PM, Andrew Wang wrote: > > Does this mean you find our current system of listing a JIRA as being fixed > in both a 2.6.x and 2.7.x to be confusing? Nope. I'm only confused when there isn't a .0 release in the fix line. When I see 2.6.x and 2.7.x I know tha

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Andrew Wang
Thanks for the input Allen, good perspective as always, inline: > From the perspective of an end user who is reading multiple > versions' listings at once, listing the same JIRA being fixed in multiple > releases is totally confusing, especially now that release notes are > actually reada

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Vinod Kumar Vavilapalli
I’ve been using jdiff simply because of a lack of alternative. If you’ve had experience with tool [1], if you think it serves our purpose, and if you can spare some time, that’ll be greatly appreciated. I can also pitch in with whatever help is needed. I think we should pick one of 2.6.3 or 2.7

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Allen Wittenauer
From the perspective of an end user who is reading multiple versions' listings at once, listing the same JIRA being fixed in multiple releases is totally confusing, especially now that release notes are actually readable. "So which version was it ACTUALLY fixed in?" is going to be the

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Andrew Wang
> > >> I am also not quite sure I understand the rationale of what's in the > HowToCommit wiki. Assuming the semantic versioning (http://semver.org) as > our baseline thinking, having concurrent release streams alone breaks the > principle. And that is *regardless of* how we line up individual rele

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-22 Thread Sangjin Lee
On Thu, Jul 21, 2016 at 3:58 PM, Andrew Wang wrote: > Thanks for the input Vinod, inline: > > > > Similarly the list of features we are enabling in this alpha would be > good > > - may be update the Roadmap wiki. Things like classpath-isolation which > > were part of the original 3.x roadmap are

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-21 Thread Andrew Wang
Thanks for the input Vinod, inline: > Similarly the list of features we are enabling in this alpha would be good > - may be update the Roadmap wiki. Things like classpath-isolation which > were part of the original 3.x roadmap are still not done. > > I already updated the website release notes at

RE: Setting JIRA fix versions for 3.0.0 releases

2016-07-21 Thread Zheng, Kai
-- From: Andrew Wang [mailto:andrew.w...@cloudera.com] Sent: Friday, July 22, 2016 3:33 AM To: Vinod Kumar Vavilapalli Cc: common-dev@hadoop.apache.org; mapreduce-...@hadoop.apache.org; yarn-...@hadoop.apache.org Subject: Re: Setting JIRA fix versions for 3.0.0 releases I really, really want

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-21 Thread Sean Busbey
On Thu, Jul 21, 2016 at 4:32 PM, Vinod Kumar Vavilapalli wrote: >> I really, really want a 3.0.0-alpha1 ASAP, since it's basically impossible >> for downstreams to test incompat changes and new features without a release >> artifact. I've been doing test builds, and branch-3.0.0-alpha1 is ready

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-21 Thread Vinod Kumar Vavilapalli
> I really, really want a 3.0.0-alpha1 ASAP, since it's basically impossible > for downstreams to test incompat changes and new features without a release > artifact. I've been doing test builds, and branch-3.0.0-alpha1 is ready for > an RC besides possibly this fix version issue. Not arguing a

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-21 Thread Sean Busbey
> Longer-term, I assume the 2.x line is not ending with 2.8. So we'd still > have the issue of things committed for 2.9.0 that will be appearing for the > first time in 3.0.0-alpha1. Assuming a script exists to fix up 2.9 JIRAs, > it's only incrementally more work to also fix up 2.8 and other unrel

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-21 Thread Andrew Wang
I really, really want a 3.0.0-alpha1 ASAP, since it's basically impossible for downstreams to test incompat changes and new features without a release artifact. I've been doing test builds, and branch-3.0.0-alpha1 is ready for an RC besides possibly this fix version issue. I'm not too worried abou

Re: Setting JIRA fix versions for 3.0.0 releases

2016-07-21 Thread Vinod Kumar Vavilapalli
The L & N fixes just went out, I’m working to push out 2.7.3 - running into a Nexus issue. Once that goes out, I’ll immediately do a 2.8.0. Like I requested before in one of the 3.x threads, can we just line up 3.0.0-alpha1 right behind 2.8.0? That simplifies most of this confusion, we can avoi