On Tue, Jun 10, 2014 at 5:19 PM, Mark McLoughlin <mar...@redhat.com> wrote: > On Tue, 2014-06-10 at 12:24 -0400, Doug Hellmann wrote: >> As part of the push to release code from the oslo incubator in >> stand-alone libraries, we have had several different discussions about >> versioning and release schedules. This is an attempt to collect all of >> the decisions we have made in those discussions and to lay out the >> rationale for the approach we are taking. I don't expect any of this >> to be a surprise, since we've talked about it, but I haven't actually >> written it all down in one place before so some of you might not have >> seen all of the points. Please let me know if you see any issues with >> the proposal or have questions. If everyone agrees that this makes >> sense, I'll put it in the wiki. > > Nice work Doug, this is looking great. > >> Background: >> >> We have two types of oslo libraries. Libraries like oslo.config and >> oslo.messaging were created by extracting incubated code, updating the >> public API, and packaging it. Libraries like cliff and taskflow were >> created as standalone packages from the beginning, and later adopted >> by the oslo team to manage their development and maintenance. >> >> Incubated libraries have been released at the end of a release cycle, >> as with the rest of the integrated packages. Adopted libraries have >> historically been released "as needed" during their development. We >> would like to synchronize these so that all oslo libraries are >> officially released with the rest of the software created by OpenStack >> developers. >> >> The first release of oslo.config was 1.1.0, as part of the grizzly >> release. The first release of oslo.messaging was 1.2.0, as part of the >> havana release. oslo.config was also updated to 1.2.0 during havana. >> All current adopted libraries have release numbers less than 1.0.0. >> >> In the past, alpha releases of oslo libraries have been distributed as >> tarballs on an openstack server, with official releases going to PyPI. >> Applications that required the alpha release specified the tarball in >> their requirements list, followed by a version specifier. This allowed >> us to prepare alpha releases, without worrying that their release >> would break continuous-deployment systems by making new library >> releases available to pip. This approach still made it difficult for >> an application developer to rely on new features of an oslo library, >> until an alpha version was produced. >> >> When the PyPI mirror was introduced in our CI system, relying on >> tarballs not available on the mirror conflicted with our desire to >> have the gate system install *only* from the package mirror. As we are >> now installing only from the mirror, we need to publish our alpha >> releases in a format that will work with the mirror. >> >> We already gate OpenStack applications and oslo libraries using >> integration tests using the normal devstack-gate jobs. During Icehouse >> we had a couple of oslo library releases that broke unit tests of >> applications after the library was released. We plan to address that >> with separate gating jobs during juno. In addition to that gating, we >> need to support developers who want to use new features of oslo >> libraries before official releases are available. >> >> A Version Numbering Scheme: >> >> At the Juno summit, Josh proposed that we use semantic versioning >> (SemVer) for oslo libraries [1]. Part of that proposal also included >> ideas for allowing breaking backwards compatibility at some release >> boundaries, and I am explicitly *not* addressing >> backwards-incompatible changes beyond saying that we do not expect to >> have any during Juno. We do need to solve the problem of breaking API >> compatibility, but I want to take one step at a time. The first step >> is choosing a rational release versioning scheme. >> >> SemVer is widely used and gives us relatively clear guidelines about >> choosing new version numbers. It supports alpha releases, which are >> going to be key to meeting some of our other requirements. I propose >> that we adopt pbr's modified SemVer [2] for new releases, beginning >> with Juno. >> >> The versions for existing libraries oslo.config and oslo.messaging >> will be incremented from their Icehouse versions but updating the >> minor number (1.x.0) at the end of the Juno cycle. >> >> All adopted libraries using numbers less than 1.0 will be released as >> 1.0.0 at the end of the Juno cycle, based on the fact that we expect >> deployers to use them in production. > > Perhaps not directly related to this proposal, but IMHO OpenStack should > not depend on any semver-versioned libraries which are still on 0.x. > > i.e. we should not depend on API unstable projects (because of the pain > it imposes on everyone), particularly ones that are likely to feel > empowered to break their API at will because of their version number :) > >> Releases during Juno should *all* be marked as alphas of the >> anticipated upcoming SemVer-based release number (1.0.0.0aN or >> 1.2.0.0aN or whatever). > > Why not simply 1.2.0aN btw?
I read Robert's pbr-semver spec (https://review.openstack.org/#/c/96608/4/specs/juno/pbr-semver.rst) to mean that we needed the extra digit to comply with PEP-440, but after looking at the PEP (http://legacy.python.org/dev/peps/pep-0440/) I don't think we actually do. > >> The new CI system can create packages as >> Python wheels and publish them to the appropriate servers, which means >> projects will no longer need to refer explicitly to pre-release >> tarballs. > > The details are a bit more nuanced here - pip won't install alpha > libraries unless you explicitly request them with a command line flag to > install any alphas available or you explicitly require the alpha > version. > > The wheels aspect is that pip <= 1.3 didn't support this, but also > didn't support wheels ... so we publish alphas only as wheels to ensure > that older pips don't see them. Thanks, I'll include those details in the final write-up. > >> Releases after Juno will follow a similar pattern, incrementing the >> minor number and using alpha releases within the development cycle. >> >> [1] https://etherpad.openstack.org/p/juno-oslo-semantic-versioning >> [2] http://docs.openstack.org/developer/pbr/semver.html >> >> Frequent Alpha Releases: >> >> While we can run gate jobs using the master branch of oslo libraries, >> developers will have to take extra steps to run unit tests this way >> locally. To reduce this process overhead, while still ensuring that >> developers use current versions of the code, we should produce alpha >> releases of libraries during the release cycle fairly frequently. I >> suggest that we start with a weekly check-up during the oslo team >> meetings, and that we tag releases early on Mondays when deemed >> necessary. >> >> Patch Releases: >> >> Updates to existing library releases can be made from stable branches. >> Checking out stable/icehouse of oslo.config for example would allow a >> release 1.3.1. We don't have a formal policy about whether we will >> create patch releases, or whether applications are better off using >> the latest release of the library. Do we need one? > > I'm not sure we need one, but if we did I'd expect them to be aligned > with stable releases. > > Right now, I think they'd just be as-needed - if there's enough > backported to the stable branch to warrant a release, we just cut one. That's pretty much what I thought, too. We shouldn't need to worry about alphas for patch releases, since we won't add features. > >> We do not typically use upper bounds on the requirements >> specifications for oslo libraries, so new releases may automatically >> be adopted by continuous-deployment systems building packages for >> stable branches of OpenStack applications. Although we have been >> careful about API compatibility in the past, there is a chance that a >> new release could break an older application. Applications could add >> an upper bound using SemVer numbering if they choose, although that >> may prevent them from seeing bug fixes. > > I think applications should only add an upper bound on the major number > so they don't see API breakage e.g. > > oslo.config>=1.2.0,<2 > > If applications have an upper bound on the minor number, that indicates > a lack of confidence in our ability to add new features without breaking > the API. > >> Unit test gate jobs: >> >> We have a blueprint for Juno to add cross-project unit test gating for >> applications and oslo libraries [3]. This will allow us to verify that >> tests for applications do not break when oslo libraries change, but >> also that those tests do not make assumptions about oslo library >> implementation details. >> >> Depending on how the job is implemented, we may be able to reuse the >> tools to let developers run the same tests locally. More thought is >> needed here; stay tuned. >> >> [3] https://blueprints.launchpad.net/oslo/+spec/enhance-cross-test-gate-job > > Very cool stuff! > > Mark. > > > _______________________________________________ > OpenStack-dev mailing list > OpenStack-dev@lists.openstack.org > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
