Sorry, in my TL;DR I forgot release frequent alphas (nightly / weekly / whatever schedule you want)
On Fri, Sep 16, 2016 at 8:18 AM Jonathan Haddad <j...@jonhaddad.com> wrote: > I've worked on a few projects where we've had a branch that new stuff went > in before merging to master / trunk. What you've described reminds me a > lot of git-flow (http://nvie.com/posts/a-successful-git-branching-model/) > although not quite the same. I'll be verbose in this email to minimize the > reader's assumptions. > > The goals of the release cycle should be (in descending order of priority): > > 1. Minimize bugs introduced through change > 2. Allow the codebase to iterate quickly > 3. Not get caught up in a ton of back porting bug fixes > > There is significant benefit to having a releasable trunk. This is > different from a trunk which is constantly released. A releasable trunk > simply means all tests should *always* pass and PMC & committers should > feel confident that they could actually put it in prod for a project that > actually matters. Having it always be releasable (all tests pass, etc) > means people can at least test the DB on sample data or evaluate it before > the release happens, and get feedback to the team when there are bugs. > > This is a different mentality from having a "features" branch, where it's > implied that at times it's acceptable that it not be stable. The > historical trend with the Cassandra codebase has been to test minimally, > throw the code over the wall, and get feedback from people putting it in > prod who run into issues. In my experience I have found a general purpose > "features" branch to result in poorly quality codebases. It's shares a lot > of the same problems as the 1+ year release cycle did previously, with > things getting merged in and then an attempt to stabilize later. > > Improving the state of testing in trunk will catch more bugs, satisfying > #1, which naturally leads to #2, and by reducing bugs before they get > released #3 will happen over time. > > My suggestion for a *supported* feature release every 3 months (could just > as well be 4 or 6) mixed with Benedict's idea of frequent non-supported > releases (tagged as alpha). Supported releases should get ~6 months worth > of bug fixes, which if done right, will decrease over time due to a > hopefully more stable codebase. I 100% agree with Mick that semver makes > sense here, it's not just for frameworks. Major.Minor.Patch is well > understood and is pretty standard throughout the world, I don't think we > need to reinvent versioning. > > TL;DR: > Release every 3 months > Support for 6 > Keep a stable trunk > New features get merged into trunk but the standard for code quality and > testing needs to be property defined as something closer to "production > ready" rather than "let the poor user figure it out" > > Jon > > > > > > > > On Fri, Sep 16, 2016 at 3:05 AM Sylvain Lebresne <sylv...@datastax.com> > wrote: > >> As probably pretty much everyone at this point, I agree the tick-tock >> experiment >> isn't working as well as it should and that it's probably worth course >> correcting. I happen to have been thinking about this quite a bit already >> as it >> turns out so I'm going to share my reasoning and suggestion below, even >> though >> it's going to be pretty long, in the hope it can be useful (and if it >> isn't, so >> be it). >> >> My current thinking is that a good cycle should accommodate 2 main >> constraints: >> 1) be useful for users >> 2) be realistic/limit friction on the development side >> and let me develop what I mean by both points slightly first. >> >> I think users mostly want 2 things out of the release schedule: they want >> a >> clearly labeled stable branch to know what they should run into >> production, >> and >> they want new features and improvements. Let me clarify that different >> users >> will want those 2 in different degrees and with variation over time, but I >> believe it's mainly some combination of those. On the development side, I >> don't >> think it's realistic to expect more than 2/3 branches/series to be >> supported at >> any one time (not going to argue that, let's call it a professional >> opinion). I >> also think accumulating new work for any meaningful length of time before >> releasing, as we used to do, is bad as it pushes devs to rush things to >> meet a >> given release deadline as they don't want to wait for the next one. This >> in >> turn >> impacts quality and creates unnecessary drama. It's also good imo to have >> a >> clear policy regarding where a given work can go (having to debate on each >> ticket on which branch it should go is a waste of dev time). >> >> With those "goals" in mind, I'll note that: >> - the fixed _and_ short cadence of tick-tock is imo very good, in >> particular in >> (but not limited to) avoiding the 'accumulate unreleased stuffs' >> problem. >> - we have ample evidence that stuffs don't get truly stable until they get >> only >> bug fixes for a few months. Which doesn't mean at all that we shouldn't >> continue to make progress on increasing the quality of new code btw. >> - Simple is also a great quality of a release cycle. I think we should try >> to >> define what's truly important to achieve (my opinion on that is above) >> and do >> the simplest thing that achieve that. This does imply the release cycle >> won't make the coffee, but that's alright, it probably shouldn't anyway. >> >> In light of all this, my suggesting for a release cycle woud be: >> - To have 3 branches: 'features', 'testing' and 'stable', with an X month >> rotation: 'features' becomes 'testing' after X months and then 'stable' >> after >> X more, before getting EOL X months later. >> - The feature branch gets everything. The testing branch only gets bug >> fixes. >> The stable branch only gets critical bug fixes. And imo, we should be >> very >> strict on this (I acknowledge that there is sometimes a bit of >> subjectivity on >> whether something is a bug or an improvement, and if it's critical or >> not, but >> I think it's not that hard to get consensus if we're all reasonable >> (though it >> might worth agreeing on some rough but written guideline upfront)). >> - We release on a short and fixed cadence of Y month(s) for both the >> feature and >> testing branch. For the stable branch, given that it already had X >> months >> of >> only bug fixes during the testing phase, one can hope critical fixes >> will >> be >> fairly rare, less than 1 per Y period on average). Further, it's >> supposed >> to >> be stable and fixes are supposed to be critical, so doing hot-fix >> releases >> probably makes the most sense (though it probably only work if we're >> indeed >> strict on what is considered critical). >> >> And that's about it. I think it would believably achieve stability (with a >> clear >> label on which releases are stable), but also provide new features and >> improvements quickly for those that wants that. The testing phase is imo a >> necessary intermediate step to get the stable one. >> >> On thing to define is X and Y. For Y (the cadence of feature/testing), I >> think 1 >> or 2 months are the only options that make sense (less than 1 month is too >> fast, >> and more than 2 months is imo starting to get too long). For X, that's >> more >> debatable but it's open-source and we should recognize volunteers >> generally >> don't want to maintain things for too long either. My 2 is that 6 or 8 >> months >> are probably the best options here. >> >> We'd also have to put some numbering scheme on top of that, but that's not >> really the important part (the meaning is in the branch labels, not the >> numbers). To give just one possible option (and assuming X=6, Y=1), in >> January >> 2017 we could cut 4.0 as the start of both 'feature' and 'testing'. We'd >> then >> have 4.1, 4.2, ... on the 'feature' branch, and 4.0.1, 4.0.2, ... on the >> testing >> branch for the next 6 months. In July, we'd switch from 4.5 to 5.0, with >> that >> becoming the new 'feature' and 'testing' base. At the same time, we'd cut >> 4.0.6 from 4.0.5 as the new 'stable' branch. Hot-fix on that stable branch >> would >> be versioned 4.0.6.1, 4.0.6.2 and so on. >> >> Of course, there can be variations on all this. We could have a different >> X >> and/or Y for the different branches to name just one. I'm just trying to >> abide >> by the "simpler is better" rule though. >> >> Lastly, I'll note that the versioning wouldn't use semver. I like semver >> alright >> but I think it's more suited to libraries than something like Cassandra. >> Amongst >> other things, it strongly rely on a fine grained notion of backward >> compatibility which I think is ill-defined for Cassandra in general. >> >> Sylvain >> >> On Fri, Sep 16, 2016 at 4:33 AM, Mick Semb Wever <m...@thelastpickle.com> >> wrote: >> >> > Totally agree with all the frustrations felt by Jon here. >> > >> > >> > TL;DR >> > Here's a proposal for 4.0 and beyond: that is puts together the comments >> > from Benedict, Jon, Tyler, Jeremy, and Ed; >> > >> > - keep bimonthly feature releases, >> > - revert from tick-tock to SemVer numbering scheme, >> > - during the release vote also vote on the quality label (feature >> branches >> > start with a 'Alpha' and the first patch release as 'Beta'), >> > - accept that every feature release isn't by default initially >> supported, >> > and its branch might never be, >> > - maintain 3 'GA' branches at any one time, >> > - accept that it's not going to be the oldest GA branches that >> necessarily >> > reach EOL first. >> > >> > >> > Background and rationale… >> > >> > IMO the problem with Tick-Tock is that it introduces two separate >> concepts: >> > - incremental development, and >> > - limiting patch releases. >> > >> > The first concept: having bimonthly tocks; made C* development more >> > incremental. A needed improvement. >> > No coincidence, at the same time as tick-tock was introduced, there was >> > also a lot of effort being put into testing and a QA framework. >> > From this we've seen a lot of fantastic features incrementally added to >> C*! >> > >> > The second concept: having bimonthly ticks; limited C* to having only >> one >> > patch release per tock release. >> > The only real benefit to this was to reduce the effort involved in >> > maintenance, required because of the more frequent tock releases. >> > The consequence is instability has gone bananas, as Jon clearly >> > demonstrates. Someone went and let the monkey out. >> > >> > A quick comparison of before to tick-tock: >> > >> > * Before tick-tock: against 6-12 months of development it took a >> > time-frame of 3-6 months and 6+ patch releases to stabilise C*. >> > >> > * After tick-tock: against 2 months of development we could have >> > expected the same time-frame of 3-6 months (because adoption is >> dictated by >> > users, not developers) and *over* this period 1-2 patch releases to >> > stabilise. It seemed to have been a fools errand to force this to 1 >> patch >> > release after only one month. It seems that the notion of incremental >> > development was applied for the developers where-as the waterfall model >> was >> > applied to QA in production for the users. (note: all this is not taking >> > into account advantages of incremental development, an improved QA >> > framework, and a move towards a stable-master.) >> > >> > The question remains to how many of these releases can the community >> afford >> > to support. And being realistic much of this effort relies upon the >> > commercial entities around the community. For example having 1 year of >> > support means having to support 6 feature releases, and there's probably >> > not the people power to do that. It also means that in effect any >> release >> > is actually only supported for 6-9 months, since it took 3-6 for it to >> get >> > to production-ready. >> > >> > A typical Apache release process is that each new major release gets >> voted >> > on as only 'Alpha' or 'Beta'. As patch releases are made it is >> ascertained >> > whether enough people are using it (eg in production) and the quality >> label >> > appropriately raised to either 'Beta' or 'GA'. The quality label can be >> > proposed in the vote or left to be voted upon by everyone. The quality >> > label is itself not part of the version number, so that the version >> number >> > can follow strict SemVer. >> > >> > Then the community can say, for example, it supports 3 'GA' branches. >> This >> > permits some major releases to never make it to GA, and others to hang >> over >> > for a bit longer. It's something that the community gets a feel for by >> > appreciating the users and actors around it. The number of branches >> > supported depends on what the community can sustain (including the new >> > non-GA branches). The community also becomes a bit more honest about the >> > quality of x.y.0 releases. >> > >> > The proposal is an example that embraces incremental development and the >> > release-often mentality, while keeping a realistic and flexible >> approach to >> > how many branches can be supported. The cost of supporting branches is >> > still very real, and pushing for a stable master means no feature >> branch is >> > cut without passing everything in the QA framework and 100% belief that >> it >> > can be put into a user's production. That is there's not a return to >> > thinking about feature branches as a place for ongoing stabilisation >> > efforts, just because they have a 'Alpha/Beta' label. The onus of work >> is >> > put upon the developer having to maintain branches for features targeted >> > for master, and not on the community having to stabilise and support >> > feature branches. >> > >> > BTW has anyone figured out whether it's the tick or the tock that >> > represents the feature release?? I probably got it wrong here :-) >> > >> > >> > ~mck >> > >> >
