Any advice on our procedure from this, Leo?
On Thu, Jul 24, 2014 at 10:39 AM, Leo Simons <lsim...@schubergphilis.com> wrote:
> On Jul 24, 2014, at 8:39 AM, Rajani Karuturi <rajani.karut...@citrix.com>
> wrote:
>> here is what i propose:
>>
>> 1. rename 'master' to 'develop’
>> 2. branch a new 'master' from '4.4’
>> 3. branch 'RELEASE-4.5' from the develop
>> 4. merge 'RELEASE-4.5' to master once the release voting is done.
>
> I like this conceptually; I’m not sure if its feasible because step 4
> requires resolving the cherry-pick mess.
>
> Ignoring awsapi (which has a lot of churn) and ignoring formatting changes,
> here are sizes of very minimalized diffs between branches
>
> branch two diff size big diff chunks
> branch one
> 4.3 4.4 11MB hyperv,netscaler,server,engine
> 4.4 4.4-forward 2MB tests,marvin
> 4.4-forward master 6MB xenapi,xen,xenserver,storage
> 4.4 master 8MB xenapi,tests,marvin,xen,xenserver
> (See below how I got the numbers.)
>
> Starting git-flow from 4.4 or 4.4-forward and then merging things in from
> master means processing hundreds of thousands of lines of diff. Of those
> lines, many many thousands of lines will probably not auto-merge due to the
> cherry-pick approach. A rebase between any of these branches is not feasible;
> git cannot un-pick what happened so it cannot offer much assistance.
>
> Looking at all the diff stats, breaking xenapi and awsapi out into their own
> git repositories (and release cycles) seems like a really good thing to do; I
> expect it will help a lot with future merges.
>
> You can try step 4. for yourself now:
>
> git checkout 4.4
> git merge master
>
> Good luck ;-)
>
>> RELEASE-4.6 branch should be off develop as all the feature branches would
>> be merged there before freeze for 4.6 and would be merged to master when the
>> release is voted.
>
> Small note, git-flow tends to use release/ prefixes for release branches.
> You'd label ‘em
>
> master
> develop
> feature/nuage-vsp-support
> feature/frob-the-foo
> support/license-header-update
> release/4.5
> release/4.6
>
>> The other question I have is in the step:4. how are we going to manage fixes
>> to 4.5 post branch cut? ( assuming no features as the freeze is done)
>
> Yes that’d be typical with git-flow. Once you create the release branch, all
> bugfixes that are to make it into that release go onto the release branch
> _first_ (and _only_ fixes go there), before being merged into develop.
>
> With git-flow you don’t really have to call it a “freeze”. Development of new
> features continues (on their feature branches) uninterrupted even if a
> release is being cut, and merging of completed features still is exactly the
> same (to develop) regardless of whether there’s active release branches.
>
> On many projects using git-flow there is no rush to try and get features
> merged before a release cut-off date, because cutting another release is easy
> and cheap that you can just do one ‘whenever’ to get your feature out there.
>
> For cloudstack that’s not quite the case due to the somewhat heavyweight
> test/release process. What I imagine is that if a feature misses a deadline,
> and the community decides it wants to include that feature anyway, is
> rebasing it onto the release branch and merging it
>
> ...decide a finished feature goes into 4.5 after all...
> git branch feature/frob-the-foo-4.5 feature/frob-the-foo
> git checkout feature/frob-the-foo-4.5
> git rebase -i release/4.5
> git push origin feature/frob-the-foo-4.5
> git checkout release/4.5
> git pull --rebase
> git merge feature/frob-the-foo-4.5
> git checkout develop
> git pull --rebase
> git merge release/4.5
> git push
> git branch -d feature/frob-the-foo-4.5
> git branch -d feature/frob-the-foo
>
> There’s little to no chance cloudstack could ever safely decide to do this
> right now, but a few months down the road, this might become safe enough,
> especially since the two explicit merge commits allow figuring out what
> happened, and possibly rolling back if the feature destabilizes the release.
>
> Your workflow would be that most tests and most QA work runs on the release
> branch, while developers switch between release branch and develop branch
> depending on what they are doing. A developer that is picking up a bug report
> associated with that release will basically
>
> ...get the bug report...
> git stash save what-I-was-doing
> git checkout release/4.5
> git pull --rebase
> ...code code code...
> ...test...
> git commit && git push
> git checkout develop
> git merge release/4.5
> git push
> git stash pop what-I-was-doing
> ...code code code...
>
> For bigger amounts of change/experimentation to work on a nasty kind of bug,
> you might have your own local temporary branch (I almost always do this
> myself):
>
> ...get the bug report...
> git stash save what-I-was-doing
> git checkout release/4.5
> git pull --ff-only
> git branch bugfix/CLOUDSTACK-42 release/4.5
> git checkout bugfix/CLOUDSTACK-42
> ...code code code...
> git commit
> ...code code code...
> git commit
> ...test...
> git pull
> git rebase -i release/4.5
> git commit
> git checkout release/4.5
> git merge bugfix/CLOUDSTACK-42
> git commit
> git checkout develop
> git merge release/4.5
> git push origin develop release/4.5
> git branch -d bugfix/CLOUDSTACK-12345
> git stash pop what-I-was-doing
> ...code code code...
>
> It’s a bit of adjustment, but once you get used to it, this lots-of-branching
> way of working is soooo much nicer than most alternatives :-)
>
>
> cheers,
>
>
> Leo
>
> PS: diff stats...
>
> $ git clean -f -x -d
>
> $ git remote -v
> lsimons g...@github.com:lsimons/cloudstack.git (fetch)
> lsimons g...@github.com:lsimons/cloudstack.git (push)
> origin git://git.apache.org/cloudstack.git (fetch)
> origin git://git.apache.org/cloudstack.git (push)
> sbpgh g...@github.com:schubergphilis/cloudstack.git (fetch)
> sbpgh g...@github.com:schubergphilis/cloudstack.git (push)
>
> # snapshot important branches
> git branch 4.3-201407231043 4.3
> git branch 4.4-201407231043 4.4
> git branch 4.4-forward-201407231043 4.4-forward
> git branch master-201407231043 master
>
> # create branches which reformat the important branches
> git branch 4.3-201407231043-format 4.3-201407231043
> git branch 4.4-201407231043-format 4.4-201407231043
> git branch 4.4-forward-201407231043-format 4.4-forward-201407231043
> git branch master-201407231043-format master-201407231043
>
> # patcher....this is pseudocode, the pom.xml patch doesn't work / apply
> cleanly
> # across branches
> function patcher() {
> patch -p1 <<END
> diff --git a/pom.xml b/pom.xml
> index 3d1e747..ad792ac 100644
> --- a/pom.xml
> +++ b/pom.xml
> @@ -463,6 +463,11 @@
> <pluginManagement>
> <plugins>
> <plugin>
> + <groupId>com.googlecode.maven-java-formatter-plugin</groupId>
> + <artifactId>maven-java-formatter-plugin</artifactId>
> + <version>0.4</version>
> + </plugin>
> + <plugin>
> <artifactId>maven-clean-plugin</artifactId>
> <configuration>
> <excludeDefaultDirectories>true</excludeDefaultDirectories>
> END
> }
>
> # trick to reformatting all java code on a branch
> function formatter() {
> git clean -f -x -d
> git checkout $1
> git clean -f -x -d
> mvn
> com.googlecode.maven-java-formatter-plugin:maven-java-formatter-plugin:0.4:format
> git add .
> git commit -m "format"
> }
>
> formatter 4.3-201407231043-format
> formatter 4.4-201407231043-format
> formatter 4.4-forward-201407231043-format
> formatter master-201407231043-format
>
> # trick to get minimalistic diff between two branches
> function minimaldiff() {
> git diff \
> \
> -U0 -M -C -w --ignore-blank-lines --find-copies-harder \
> --diff-algorithm=minimal \
> $1..$2 | egrep -v '^[+-] *#'
> }
>
> minimaldiff 4.3-201407231043-format 4.4-201407231043-format | >
> ~/4.3-to-4.4.txt
> 4.3-to-4.4.txt is a 31MB text file
>
> $ git diff --dirstat=lines,2,cumulative 4.3-201407231043-format
> 4.4-201407231043-format
> 37.9% awsapi/src/com/amazon/ec2/client/
> 76.5% awsapi/src/com/amazon/ec2/
> 3.5% awsapi/src/com/amazon/s3/client/
> 7.7% awsapi/src/com/amazon/s3/
> 84.2% awsapi/src/com/amazon/
> 2.5% awsapi/src/com/cloud/
> 86.7% awsapi/src/com/
> 86.7% awsapi/
> 4.3% plugins/
>
> 85% of the diff is in awsapi, so lets filter that out...
>
> git filter-branch \
> --index-filter 'git rm -r --cached --ignore-unmatch awsapi' \
> 4.3-201407231043-format \
> 4.4-201407231043-format \
> 4.4-forward-201407231043-format \
> master-201407231043-format
>
> minimaldiff 4.3-201407231043-format 4.4-201407231043-format >
> ~/4.3-to-4.4-no-aws.txt
> 4.3-to-4.4-no-aws.txt is a 11MB text file
>
> $ git diff --dirstat=lines,4 4.3-201407231043-format 4.4-201407231043-format
> 5.4% api/src/org/apache/cloudstack/api/command/
> 4.0% deps/XenServerJava/src/com/xensource/xenapi/
> 6.4% engine/
> 4.1% plugins/hypervisors/hyperv/DotNet/ServerResource/
> 10.3% plugins/hypervisors/
> 4.3% plugins/network-elements/netscaler/src/com/cloud/
> 10.0% plugins/network-elements/
> 8.1% server/src/com/cloud/
> 5.1% services/secondary-storage/
> 4.9% ui/scripts/
> 4.7% ui/
> 4.1% vmware-base/src/com/cloud/hypervisor/vmware/mo/
>
> minimaldiff 4.4-201407231043-format 4.4-forward-201407231043-format >
> ~/4.4-to-4.4.1-no-aws.txt
> ~/4.4-to-4.4.1-no-aws.txt is a 2MB text file
>
> $ git diff --dirstat=lines,2 4.4-201407231043-format
> 4.4-forward-201407231043-format
> 2.8% plugins/hypervisors/hyperv/DotNet/ServerResource/WmiWrappers/
> 2.4% plugins/hypervisors/
> 47.6% test/integration/component/
> 8.1% test/integration/smoke/
> 2.8% tools/marvin/marvin/config/
> 9.4% tools/marvin/marvin/integration/lib/
> 11.7% tools/marvin/marvin/lib/
> 6.9% tools/marvin/marvin/
> 2.2% ui/scripts/
>
> minimaldiff 4.4-201407231043-format master-201407231043-format >
> ~/4.4.0-to-4.5-no-aws.txt
> ~/4.4.0-to-4.5-no-aws.txt is a 7.7MB text file
>
> $ git diff --dirstat=lines,4 4.4-201407231043-format
> master-201407231043-format
> 26.5% deps/XenServerJava/src/com/xensource/xenapi/
> 9.8% plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/
> 9.7%
> plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resourc
> 4.9% plugins/hypervisors/
> 7.6% plugins/
> 13.1% test/integration/component/
> 8.8% tools/marvin/marvin/
>
> minimaldiff 4.4-forward-201407231043-format master-201407231043-format >
> ~/4.4.1-to-4.5-no-aws.txt
> ~/4.4.1-to-4.5-no-aws.txt is a 5.6MB text file
>
> $ git diff --dirstat=lines,4 4.4-forward-201407231043-format
> master-201407231043-format
> 35.4% deps/XenServerJava/src/com/xensource/xenapi/
> 13.1% plugins/hypervisors/xen/src/com/cloud/hypervisor/xen/resource/
> 13.0%
> plugins/hypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resourc
> 4.6% plugins/hypervisors/
> 4.2% plugins/network-elements/
> 5.1% plugins/storage/volume/
> 4.1% scripts/
> 4.1% server/
>
--
Daan
