> On 10 Sep 2015, at 18:31, Robert Kowalski <r...@kowalski.gd> wrote: > > Hi Jan, > > how would testing of a PR/change work in practice? > > Example: I am opening 3 Pull-Requests that depend on each other, one in > chttpd, one in fabric and one in rexi. > > Or is the idea that the testsuite turns red when the deps are updated in > the top level module?
Yeah, if the top module only works with all three PRs in concert, you get two broken builds and then a working build. (some of the details of what I explained earlier might make it sound that this isn’t possible, rest assured, we’ve considered this case :) Best Jan -- > > On Thu, Sep 10, 2015 at 2:09 PM, Jan Lehnardt <j...@apache.org> wrote: > >> >>> On 10 Sep 2015, at 12:18, Alexander Shorin <kxe...@gmail.com> wrote: >>> >>> On Thu, Sep 10, 2015 at 12:40 PM, Dale Harvey <d...@arandomurl.com> >> wrote: >>>> I dont think CI is a dream, It should take an afternoon at most to get >>>> CouchDB setup on travis on a single platform to ensure no major >> regressions >>>> come though. If anyone wants help doing that feel free to ping me on >>>> #pouchdb / #couchdb, we already test couchdb master in travis. >>> >>> It is a dream for now. >>> >>> We have travis enabled, but it has practical flaw: it cannot handle >>> changes that affected multiple repositories at the same time. Also, >>> for each repository we have to build full CouchDB to test what is a >>> bit overkill. >>> >>>> I do think multirepos is an issue and that solution is not so simple, we >>>> went through the same issues with pouchdb as we attempted to split out >> our >>>> repository. >>> >>> Well, there was a good theory that each of our repository will contain >>> an app which could be reused outside of CouchDB or being easily >>> replaced by alternative implementation. Suddenly, for now most of our >>> apps are heavy coupled >> >> There are two distinct functions that git does for us here: >> 1. Separate code into different modules. >> 2. Make a module available to downstream users. >> >> In a Node/npm, Python/pip, Ruby/Gems world, 1. is solved by git, 2. is >> solved by npm/pip/Gems. Is there a way to publish an Erlang module for >> downstream users >> independently of git? If yes, we could move back to a single-git-tree >> model for >> CouchDB, but we can still release parts as independent modules. I know >> there have >> been a few attempts at Erlang package managers, but has anything taken off? >> >> * * * >> >> For Hoodie, we came up with this, based on our experience with using the >> full top level test suite on each module (it is indeed overkill): >> >> - separate modules >> - one module that is the top level module (like our couchdb.git) that only >> includes module dependencies and a top-level integration test suite. >> - two release trains: stable and next. >> - a service that attempts to create a new version of the top level module >> with every commit to a dependent module. E.g. we automatically bump the >> dependency in a checkout of the top level module, run the integration >> tests, if they fail, nothing happens and we report that back on the commit >> to the dependent module. If it succeeds, we automatically determine the >> next version number on the next release train (SemVer makes that easy) and >> release the new top level module with the updated dependency. >> - once we are comfortable that the next release train is stable enough for >> a release, we change switch out the release trains: next becomes stable, >> and we have a new empty next train (npm uses release tags for this, but the >> mechanics are irrelevant, although something like that needs to be >> supported in the package manager). >> >> This requires: >> - Strict SemVer on all modules (we use >> https://github.com/semantic-release/semantic-release) to make sure humans >> can’t screw this up. >> - Breaking change detection, so we don’t accidentally release a BC break >> in a non-major version (available as semantic-release plugins). >> - the service that does the auto-update dance described above. >> >> This is a lot of infrastructure (and some of it we are still developing), >> but once in place, it is really simple to not screw this up and it takes a >> huge cognitive load off of everyday development. >> >> Semantic-Release is language agnostic in theory, but afaik there are only >> Node and Python versions, so we’d have to find someone to make an Erlang >> variant. Also, the process uses npm heavily, so some sort of package >> management system would be nice. >> >> Eventually, I’d love to see this for CouchDB. >> >> PS: The person who invented semantic-release works for my company, and I’d >> be more than happy to have them have a crack at this. If anyone wants to >> sponsor an erlang-semantic-release, please get in touch. >> >> * * * >> >> Either way, this is a bit of a long shot. For now: >> >> We should do what Dale proposed in 2.: pin top level dependencies. I >> believe that was the plan all along, but with a fast moving 2.0 we just >> didn’t do it yet. Now is the time to pin versions and manually upgrade them >> after thorough testing (can still be in PRs and tested on Travis and >> everything). >> >> That means all of these modules (and their sub-deps) should get releases >> and release tags and those tags need to be filled into our top-level >> rebar.config.script: >> >> https://github.com/apache/couchdb/blob/master/rebar.config.script#L19-L53 >> >> Let’s get going :) >> >> Thanks >> Jan >> -- >> >> >> >> >>> >>>> 1. Dont split out into lots of repositories, if you put those components >>>> inside the CouchDB repo, then they will get the CouchDB tests run >> against >>>> them when changes are made and you wont break the CouchDB repo. >>> >>> It's good in retrospective, but useless now. We have to decide how our >>> CI will work: with multirepo layout or first fold all core repos back >>> to the one. I think we are ready to revise this too and probably this >>> should be done before CI work. >>> >>>> 2. Anything that does live outside the CouchDB repo, pin their version >>>> inside the CouchDB repo, dont have commits to subproject X >> automatically be >>>> applied to CouchDB, that means you can commit whatever you want to X but >>>> CouchDB will still be working, when you come to update the version of X >> you >>>> pinned, you can see that it breaks and not update until it is fixed. >>> >>> It's good strategy for milestones or when you don't have to manage 40+ >> repos (: >>> >>>> 2.5 Another strategy that can help with this is to have the full >> CouchDB >>>> test suite run inside X, so changes to X will run the full CouchDB suite >>>> with an updated X >>> >>> This works, but until changes are belongs to a single repository. See >>> above about travis issue. >>> >>> >>> P.S. Jason, you are right again (: >>> >>> -- >>> ,,,^..^,,, >> >> -- >> Professional Support for Apache CouchDB: >> http://www.neighbourhood.ie/couchdb-support/ >> >> -- Professional Support for Apache CouchDB: http://www.neighbourhood.ie/couchdb-support/
