+1 to semver, shippable trunk, feature flags, and better documentation about
feature support and compatibility edges - we should have a single page with a
table of version x feature, with a summary and links to detailed explanations
of everything important a user should be aware of.
I didn't really understand the unreleased versions proposal though.
On 30/04/2021, 20:00, "Joshua McKenzie" <jmcken...@apache.org> wrote:
Success of this seems predicated on keeping trunk shippable, which means
getting Harry and things like adelphi into CI/CD sooner rather than later.
I've definitely come around to Ariel's thinking about the value of having
an *actually* shippable trunk + the value of feature flags for things. It's
a heavy lift to ensure shippability with something as complex as Cassandra,
but if we can figure that out it'll go a long way towards us being able to
move forward faster as a project while remaining safe.
+1
On Fri, Apr 30, 2021 at 2:52 PM Jeremiah D Jordan
<jeremiah.jor...@gmail.com>
wrote:
> +1 for doing this (or something similar). It will give more clarity to
> downstream users about the compatibility of a given release.
>
> -Jeremiah
>
> > On Apr 30, 2021, at 12:45 PM, Mick Semb Wever <m...@apache.org> wrote:
> >
> > *** Proposal ***
> > Aligned to the agreed-upon annual cadence of supported releases, let's
> > use semantic versioning for better ecosystem operatibility, and to
> > promote API awareness and compatibility support from documentation to
> > tests.
> >
> >
> > *** Background ***
> > The recent¹ dev ML thread 'Releases after 4.0' landed on an annual
> > release cadence, and for promoting an always shippable trunk (repeated
> > again in the roadmap thread²).
> >
> > A digression that occurred in the thread was around the use of
> > semantic versioning, and the possible role of properly using major and
> > minor versions within the annual release cycle. This proposal is an
> > attempt to take those points of view and build them on everything else
> > we have agreed upon so far.
> >
> >
> > *** Ecosystem Operability ***
> > The Cassandra codebase has an ecosystem around it. From downstream
> > projects to vendors providing support for versions to managed DBaaS.
> >
> > We can help them out with semver, and by providing unreleased minor
> > versions through the year. Unreleased means we don’t do a formal
> > Apache release approval, we just bump the version in `build.xml`.
> > Downstream projects face overhead when, either trying to keep up with
> > trunk through each annual development cycle, or trying to rebase
> > against a whole year's worth of development once each year.
> > Unreleased versions will provide safe points for the ecosystem to plug
> > into and keep up with. Vendors are also free to support and provide
> > hot-fixes and back ports on these unreleased versions, outside of the
> > community's efforts or concerns. And of course semver provides a lot
> > of value to downstream codebases.
> >
> >
> > *** API and Compatibility Awareness ***
> > The idea here is to provide awareness and improved documentation to
> > our APIs, their audience, and to what compatibility is required on
> > them. Personally, I still struggle getting my head around all the ways
> > Cassandra can break its APIs and what to think about and to test when
> > coding.
> >
> > This is important for ensuring availability during upgrades
> > (mix-version clusters), and again important if we want to introduce
> > data-safe downgrades. This stuff doesn't get (battle-) tested enough.
> > The native protocol bump to v6 was an example for the need to be
> > better at documenting and testing what's involved (across the
> > ecosystem).
> >
> > The consequences of breaking compatibility range from documentation,
> > and tests, to mixed versioned clusters, upgrade and rollback
> > operations. Semantic versioning is a way of foreseeing and preparing
> > for such changes. In practice this can be done
> > a) using different fixVersions in jira ticket, and
> > b) lazy-incrementing the major version in trunk when the first
> > breaking change lands in the development cycle.
> >
> > For example, we enter the next development cycle with Jira fixVersions
> > of "4.X" and "5.X", and an initial trunk version of "4.1". Then when a
> > committer merges the first "5.X" ticket they bump trunk's version up
> > to "5.0".
> >
> > This approach incentivises patches to be aware and to better document
> > the breakage, and comes with the added benefit for the ecosystem of
> > identifying where in the development cycle the compatibility first
> > broke.
> >
> > Some examples of compatibility areas are CQL, Native Protocol, gossip,
> > JMX, Metrics, Virtual Tables, SSTable, CDC, Commitlog, FQL, and
> > Auditing. Many of these don't have enough documentation of how they
> > are versioned and compatibility. As we add pluggability (i.e. SPIs)
> > both the need to document this, and to be closer with the ecosystem
> > increases.
> >
> >
> > *** Example for 2021-2022 ***
> > Illustrating this in action, with a cadence of a minor version every
> quarter,
> >
> > - today, we branch `cassandra-4.0` and increment trunk to 4.1
> > - commits roll into trunk, no "5.X" tickets have landed yet,
> > - in July we increment the version to 4.2, no release is made or
> announced,
> > - commits continue to roll into trunk, still no "5.X" tickets have
> landed yet,
> > - in October we increment the version to 4.3
> > - commits continue to roll into trunk, a "5.X" patch lands, trunk is
> > incremented to 5.0
> > - in January 2022 we increment the value to 5.1, no release is made or
> > announced,
> > - commits continue to roll into trunk,
> > - in April 2022 we formally release 5.1 and branch `cassandra-5.1`
> >
> >
> > The cadence of those minor versions could be anything, quarterly,
> > monthly or on-demand. This practice will force us to organise and
> > automate dealing with version changes, creating our release branches,
> > organising our test upgrade version paths. I'm gathering that process
> > currently in CASSANDRA-16642.
> >
> > Jeremiah originally (and in more depth) illustrated this here:
> >
>
https://lists.apache.org/thread.html/r9b53342e6992cf98e8b95e763f63d19c798765be3bd86436f07afa8c%40%3Cdev.cassandra.apache.org%3E
> >
> >
> > *** Concerns ***
> > Addressing the questions and concerns that were previously raised.
> >
> > We have a problematic history with release versioning. This proposal
> > is not tick-tock. It is about known best-practices around semver
> > version numbers. This does not add the overhead of additional releases
> > or release branches to the community.
> >
> > Long development cycles with only a (major) release every year will be
> > an opposite force to our efforts to maintain an always shippable
> > trunk. Semver, closer and more frequent feedback from the ecosystem,
> > and better API awareness, all help us maintain an always shippable
> > trunk. This was touched on by Benedict's "quarterly 'develop'
> > releases" and by Benjamin's "bleeding edge snapshots where we do not
> > guarantee stability".
> >
> > Individual features, new and old, still can be marked with their own
> > maturity-state flag, e.g. experimental, unstable, stable, deprecated.
> > This is all aside to semver, though it is part of, and feeds into, the
> > API awareness. Deprecating and removing individual features should be
> > easier too, as their lifecycle avoids being tied to the annual
> > releases.
> >
> > "Our major/minor history has been a meaningless distinction". This
> > proposal is an attempt to fix that. With better API awareness, and a
> > way to appease the ecosystem getting what they need sooner, I believe
> > it will help us better limit what we put into our patch releases.
> >
> > Could we cut releases off such quarterly minor releases but not
> > maintain them? This was the general proposition in the previous
> > thread, and while it is possible, and would leave such unsupported
> > releases in an easy to download location with the ASF, it is left out
> > for simplicity's sake. All downstreams can use the minor versions
> > easily enough with or without a formal ASF compliant release. But it
> > is something we can add in the future if called for and we have the
> > bandwidth for. It could also be possible to better stage our
> > development builds (using nexus, artifactory, etc).
> >
> >
> > *** Summary ***
> > I'm creating the cassandra-4.0 branch and will bump trunk to version
> > "4.1" for now, until the discussion lands… I'm sure there's other
> > concerns and suggestions.
> > I can also write this up as a CEP if that's called for.
> >
> >
> >
> > [1]
>
https://lists.apache.org/thread.html/re15543b55e5d01245ad75f7ec35af97e9895d37c01562eab31963dd4%40%3Cdev.cassandra.apache.org%3E
> > [2]
>
https://lists.apache.org/thread.html/r611316edc1c6b8d331994b4625c1a4d52ae5d5aee0bf4a158b2618ba%40%3Cdev.cassandra.apache.org%3E
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> > For additional commands, e-mail: dev-h...@cassandra.apache.org
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
> For additional commands, e-mail: dev-h...@cassandra.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@cassandra.apache.org
For additional commands, e-mail: dev-h...@cassandra.apache.org
