The problem with long release cycles is that everything goes in. and you
have potentially a mish-mash of features, some more done than others,
causing instability. Quick releases attempt to fix this issue by keeping
the number of commits down to a manageable size. The problem is that
that commit list isn't necessarily cohesive and so you really haven't
solved anything, most likely.
It seems to me, the answer is to move towards feature branches, without
any concept of release branches. When a feature is truely done, and
people and tests are happy, you merge it to a newly created release
branch, and ship it, by itself. In that way the release branch doesn't
have flotsum/jetsum in it.
This of course stresses the CI environment as now you are going to ask
to build n branches at once, but not that big of a deal.
---
<br type="_moz" />
On 2017-01-10 11:13, Ariel Weisberg wrote:
Hi,
With yearly releases trunk is going to be a mess when it comes time to
cut a release. Cutting releases is when people start caring whether all
the things in the release are in a finished state. It's when the state
of CI finally becomes relevant.
If we wait a year we are going to accumulate a years worth of
unfinished
stuff in a single release. It's more expensive to context switch back
and then address those issues. If we put out large unstable releases it
means time until the features in the release are usable is pushed back
even further since it takes another 6-12 months for the release to
stabilize. Features introduced at the beginning of the cycle will have
to wait 18-24 months before anyone can benefit from them.
Is the biggest pain point with tick-tock just the elimination of long
term support releases? What is the pain point around release frequency?
Right now people should be using 3.0 unless they need a bleeding edge
feature from 3.X and those people will have to give up something to get
something.
Ariel
On Tue, Jan 10, 2017, at 10:29 AM, Jonathan Haddad wrote:
I don't see why it has to be one extreme (yearly) or another
(monthly).
When you had originally proposed Tick Tock, you wrote:
"The primary goal is to improve release quality. Our current major
“dot
zero” releases require another five or six months to make them stable
enough for production. This is directly related to how we pile
features
in
for 9 to 12 months and release all at once. The interactions between
the
new features are complex and not always obvious. 2.1 was no
exception,
despite DataStax hiring a full tme test engineering team specifically
for
Apache Cassandra."
I agreed with you at the time that the yearly cycle was too long to be
adding features before cutting a release, and still do now. Instead
of
elastic banding all the way back to a process which wasn't working
before,
why not try somewhere in the middle? A release every 6 months (with
monthly bug fixes for a year) gives:
1. long enough time to stabilize (1 year vs 1 month)
2. not so long things sit around untested forever
3. only 2 releases (current and previous) to do bug fix support at any
given time.
Jon
On Tue, Jan 10, 2017 at 6:56 AM Jonathan Ellis <jbel...@gmail.com>
wrote:
> Hi all,
>
> We’ve had a few threads now about the successes and failures of the
> tick-tock release process and what to do to replace it, but they all died
> out without reaching a robust consensus.
>
> In those threads we saw several reasonable options proposed, but from my
> perspective they all operated in a kind of theoretical fantasy land of
> testing and development resources. In particular, it takes around a
> person-week of effort to verify that a release is ready. That is, going
> through all the test suites, inspecting and re-running failing tests to see
> if there is a product problem or a flaky test.
>
> (I agree that in a perfect world this wouldn’t be necessary because your
> test ci is always green, but see my previous framing of the perfect world
> as a fantasy land. It’s also worth noting that this is a common problem
> for large OSS projects, not necessarily something to beat ourselves up
> over, but in any case, that's our reality right now.)
>
> I submit that any process that assumes a monthly release cadence is not
> realistic from a resourcing standpoint for this validation. Notably, we
> have struggled to marshal this for 3.10 for two months now.
>
> Therefore, I suggest first that we collectively roll up our sleeves to vet
> 3.10 as the last tick-tock release. Stick a fork in it, it’s done. No
> more tick-tock.
>
> I further suggest that in place of tick tock we go back to our old model of
> yearly-ish releases with as-needed bug fix releases on stable branches,
> probably bi-monthly. This amortizes the release validation problem over a
> longer development period. And of course we remain free to ramp back up to
> the more rapid cadence envisioned by the other proposals if we increase our
> pool of QA effort or we are able to eliminate flakey tests to the point
> that a long validation process becomes unnecessary.
>
> (While a longer dev period could mean a correspondingly more painful test
> validation process at the end, my experience is that most of the validation
> cost is “fixed” in the form of flaky tests and thus does not increase
> proportionally to development time.)
>
> Thoughts?
>
> --
> Jonathan Ellis
> co-founder, http://www.datastax.com
> @spyced
>
