Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-10 Thread Allen Wittenauer
On Sep 10, 2014, at 6:17 PM, Allen Wittenauer wrote: > Here’s a (merged) 3.x changes.txt file from the current output built off of > JIRA. (The unmerged versions are also created, but look pretty much > identical.) > > http://pastebin.com/fEUKSDG4 Woops, that’s only common. Here’s the m

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-10 Thread Allen Wittenauer
Fixed the subtasks. Hacked on my changes generator a bit more. Source is in my github repo if anyone wants to play with it. Here’s a (merged) 3.x changes.txt file from the current output built off of JIRA. (The unmerged versions are also created, but look pretty much identical.) http://pas

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-10 Thread Allen Wittenauer
FWIW, I’ve gone through a 2nd pass of JIRA for 3.0.0. I’m reasonable confident that as of this very second that the 155 non-sub-tasks are only in trunk. I did not do a pass over the 35 sub-tasks claiming to be in 3.x. Fun fact: the oldest patch is from 2011 and turned 3 years old this month.

Re: migrating private branches to the new git repo

2014-09-08 Thread Colin McCabe
+1 for using git log instead of CHANGES.txt. Colin On Wed, Sep 3, 2014 at 11:07 AM, Chris Douglas wrote: > On Tue, Sep 2, 2014 at 2:38 PM, Andrew Wang wrote: >> Not to derail the conversation, but if CHANGES.txt is making backports more >> annoying, why don't we get rid of it? It seems like we

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-05 Thread Allen Wittenauer
On Sep 5, 2014, at 9:19 AM, Karthik Kambatla wrote: > On Thu, Sep 4, 2014 at 8:37 AM, Allen Wittenauer wrote: >> >>We do need to have a talk about 3.x though. Looking over the >> list, it would appear that a lot of (what became) early 2.x JIRAs were >> marked as Fixed against 3.x. Pr

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-05 Thread Karthik Kambatla
On Thu, Sep 4, 2014 at 8:37 AM, Allen Wittenauer wrote: > > I hacked on it some more and yes, it’s very easy to detect: > > * type of jira (improvement, bug, new feature, wish, task, sub-task) > * incompatible or not (regardless of type) > * reviewed or not (regardless of type) > >

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-04 Thread Allen Wittenauer
I hacked on it some more and yes, it’s very easy to detect: * type of jira (improvement, bug, new feature, wish, task, sub-task) * incompatible or not (regardless of type) * reviewed or not (regardless of type) A key question is what to do about tasks, sub-tasks, and wishes. I

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-04 Thread Steve Loughran
is there any way of isolating compatible/incompatible changes, & new features? I know that any change is potentially incompatible —but it is still good to highlight the things we know are likely to cause trouble On 4 September 2014 02:51, Allen Wittenauer wrote: > Nothing official or clean or

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
Oh, it's in hdfs. Sneaky. On Sep 3, 2014, at 7:10 PM, Allen Wittenauer wrote: > > I don't see HADOOP-10957 in hadoop-common-project/hadoop-cmmon/CHANGES.txt on > github in the 2.5.1 branch. > > On Sep 3, 2014, at 7:00 PM, Karthik Kambatla wrote: > >> 2.5.1 - I believe all the commits here

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
I don't see HADOOP-10957 in hadoop-common-project/hadoop-cmmon/CHANGES.txt on github in the 2.5.1 branch. On Sep 3, 2014, at 7:00 PM, Karthik Kambatla wrote: > 2.5.1 - I believe all the commits here are in CHANGES.txt of 2.5.1. Which > one is missing? > > > On Wed, Sep 3, 2014 at 6:51 PM, Al

Re: auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-03 Thread Karthik Kambatla
2.5.1 - I believe all the commits here are in CHANGES.txt of 2.5.1. Which one is missing? On Wed, Sep 3, 2014 at 6:51 PM, Allen Wittenauer wrote: > Nothing official or clean or whatever, but just to give people an idea of > what an auto generated CHANGES.txt file might look like, here are some

auto-generating changes.txt was: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
Nothing official or clean or whatever, but just to give people an idea of what an auto generated CHANGES.txt file might look like, here are some sample runs of the hacky thing I built, based upon the fixVersion information. It doesn't break it down by improvement, etc. Also, the name on the en

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
On Sep 3, 2014, at 4:57 PM, Chris Douglas wrote: > On Wed, Sep 3, 2014 at 11:42 AM, Allen Wittenauer wrote: >> We’ll also need to get much more strict about Fix Version really only >> listing the earliest version. Many of list (next release) + (trunk), myself >> included, which after looki

Re: migrating private branches to the new git repo

2014-09-03 Thread Karthik Kambatla
Sorry, if I wasn't clear. I find Target and Fix versions useful. I find CHANGES.txt redundant. On Wed, Sep 3, 2014 at 2:55 PM, Karthik Kambatla wrote: > > On Tue, Sep 2, 2014 at 2:38 PM, Andrew Wang > wrote: > >> Not to derail the conversation, but if CHANGES.txt is making backports >> more >>

Re: migrating private branches to the new git repo

2014-09-03 Thread Chris Douglas
On Wed, Sep 3, 2014 at 11:42 AM, Allen Wittenauer wrote: > We’ll also need to get much more strict about Fix Version really only listing > the earliest version. Many of list (next release) + (trunk), myself > included, which after looking through some of the commit docs, is not correct. Is thi

Re: migrating private branches to the new git repo

2014-09-03 Thread Sandy Ryza
I personally find Target Version to be useful for tracking which version a change is intended for. On Wed, Sep 3, 2014 at 2:55 PM, Karthik Kambatla wrote: > On Tue, Sep 2, 2014 at 2:38 PM, Andrew Wang > wrote: > > > Not to derail the conversation, but if CHANGES.txt is making backports > more

Re: migrating private branches to the new git repo

2014-09-03 Thread Karthik Kambatla
On Tue, Sep 2, 2014 at 2:38 PM, Andrew Wang wrote: > Not to derail the conversation, but if CHANGES.txt is making backports more > annoying, why don't we get rid of it? It seems like we should be able to > generate it via a JIRA query, and "git log" can also be used for a quick > check (way faste

Re: migrating private branches to the new git repo

2014-09-03 Thread Arpit Agarwal
Not sure it is sufficient reason to keep the field around given the confusion it causes. :-) Thanks for volunteering to cleanup the existing Jiras. On Wed, Sep 3, 2014 at 1:09 PM, Allen Wittenauer wrote: > > I always used target version as "I hope to get this patch in by then". :D > > > On Sep

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
I always used target version as "I hope to get this patch in by then". :D On Sep 3, 2014, at 1:07 PM, Arpit Agarwal wrote: > Thanks, good to know. > > So in the spirit of simplification can we just eliminate the 'Target > Version' field from Jira? > > > On Wed, Sep 3, 2014 at 1:01 PM, Allen

Re: migrating private branches to the new git repo

2014-09-03 Thread Arpit Agarwal
Thanks, good to know. So in the spirit of simplification can we just eliminate the 'Target Version' field from Jira? On Wed, Sep 3, 2014 at 1:01 PM, Allen Wittenauer wrote: > I was doing that too, but I went "to the source": > > https://wiki.apache.org/hadoop/HowToCommit says: > > "Resolve the

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
I was doing that too, but I went "to the source": https://wiki.apache.org/hadoop/HowToCommit says: "Resolve the issue as fixed, thanking the contributor. Always set the "Fix Version" at this point, but please only set a single fix version, the earliest release in which the change will appear. S

Re: migrating private branches to the new git repo

2014-09-03 Thread Arpit Agarwal
Allen, I think the correct field you are looking for is 'Target Version'. If a fix is committed to both branch-2 and trunk, the fix version must include both 2.x.0 and 3.0.0. However the target version would be 2.x.0 as that is the earliest release that includes the fix. On Wed, Sep 3, 2014 at 12

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
Figured it out. Basically can only do bulk fix version edits of one project at a time since the versions are technically different for every project. i.e.,: you can edit all of YARN-xxx, but you can not do YARN-xxx and HDFS-xxx together. FWIW, there are 372 issues that have a fixVersion for

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
OK, it does, but only under certain conditions. Hmm. On Sep 3, 2014, at 12:04 PM, Allen Wittenauer wrote: > > > Looks like the web UI doesn't allow for bulk change of Fix Version. > > *cries* > > On Sep 3, 2014, at 11:56 AM, Andrew Wang wrote: > >> Allen, if you're willing to do the legwo

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
Looks like the web UI doesn't allow for bulk change of Fix Version. *cries* On Sep 3, 2014, at 11:56 AM, Andrew Wang wrote: > Allen, if you're willing to do the legwork, that'd be great. I feel we > should start by getting a JIRA script checked into dev-support that > generates a changelog fo

Re: migrating private branches to the new git repo

2014-09-03 Thread Andrew Wang
Allen, if you're willing to do the legwork, that'd be great. I feel we should start by getting a JIRA script checked into dev-support that generates a changelog for a release. We could then use that to make sure the various fields are set properly for previous releases, remove CHANGES.txt once we'r

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
On Sep 3, 2014, at 11:42 AM, Allen Wittenauer wrote: > > On Sep 3, 2014, at 11:07 AM, Chris Douglas wrote: > >> >> As long as release notes and incompatible changes are recorded in each >> branch, we gain no accuracy by maintaining this manually. Commit >> messages that record the merge revi

Re: migrating private branches to the new git repo

2014-09-03 Thread Allen Wittenauer
On Sep 3, 2014, at 11:07 AM, Chris Douglas wrote: > > As long as release notes and incompatible changes are recorded in each > branch, we gain no accuracy by maintaining this manually. Commit > messages that record the merge revisions instead of the change are > similarly hateful. -C We’ll als

Re: migrating private branches to the new git repo

2014-09-03 Thread Chris Douglas
On Tue, Sep 2, 2014 at 2:38 PM, Andrew Wang wrote: > Not to derail the conversation, but if CHANGES.txt is making backports more > annoying, why don't we get rid of it? It seems like we should be able to > generate it via a JIRA query, and "git log" can also be used for a quick > check (way faster

Re: migrating private branches to the new git repo

2014-09-03 Thread Steve Loughran
On 3 September 2014 02:47, Todd Lipcon wrote: > On Tue, Sep 2, 2014 at 2:38 PM, Andrew Wang > wrote: > > > Not to derail the conversation, but if CHANGES.txt is making backports > more > > annoying, why don't we get rid of it? It seems like we should be able to > > generate it via a JIRA query,

Re: migrating private branches to the new git repo

2014-09-02 Thread Todd Lipcon
On Tue, Sep 2, 2014 at 2:38 PM, Andrew Wang wrote: > Not to derail the conversation, but if CHANGES.txt is making backports more > annoying, why don't we get rid of it? It seems like we should be able to > generate it via a JIRA query, and "git log" can also be used for a quick > check (way faste

Re: migrating private branches to the new git repo

2014-09-02 Thread Andrew Wang
Not to derail the conversation, but if CHANGES.txt is making backports more annoying, why don't we get rid of it? It seems like we should be able to generate it via a JIRA query, and "git log" can also be used for a quick check (way faster than svn log). On Tue, Sep 2, 2014 at 12:38 PM, Steve Lou

Re: migrating private branches to the new git repo

2014-09-02 Thread Andrew Wang
> > Maybe at the end add the action I will ultimately take (git branch remote > rename asflive origin && git branch rename live/trunk trunk) > > +1, that'd be great > > > Somewhat related, it'd be nice to update the GitAndHadoop instructions on > > how to generate a patch using git-format-patch. I

Re: migrating private branches to the new git repo

2014-09-02 Thread Steve Loughran
I've now done my first commits; one into trunk (10373), one into branch-2 and cherry picked (fix in hadoop-common-project/hadoop-common/src/main/native/README ; no JIRA). I made an initial attempt to cherry pick the HADOOP-10373 patch from trunk into branch-2, with CHANGES.TXT being a dramatic eno

Re: migrating private branches to the new git repo

2014-09-02 Thread Steve Loughran
On 2 September 2014 19:01, Andrew Wang wrote: > This is basically what I did, make patches of each of my branches and then > reapply to the new trunk. One small recommendation would be to make the > remote named "apache" rather than "asflive" so it's consistent with the > GitAndHadoop wikipage. I

Re: migrating private branches to the new git repo

2014-09-02 Thread Andrew Wang
This is basically what I did, make patches of each of my branches and then reapply to the new trunk. One small recommendation would be to make the remote named "apache" rather than "asflive" so it's consistent with the GitAndHadoop wikipage. IMO naming branches with a "/" (e.g. "live/trunk") is als

migrating private branches to the new git repo

2014-09-02 Thread Steve Loughran
Now that hadoop is using git, I'm migrating my various work-in-progress branches to the new commit tree 1. This is the process I've written up for using git format-patch then git am to export the patch sequence and merge it in, then rebasing onto trunk to finally get in sync https://wiki.apache.