Greg Hogan <c...@greghogan.com> writes: > How "ready" do branches need to be in the "request to merge" queue?
The relevant bit is: To help coordinate the merging of branches, you must create a new guix-patches issue each time you create a branch (see The Issue Tracker). From [1], the rational was to combine creating the issue with creating the branch to avoid the issue being missed, but I forget the specifics that motivated the change. 1: https://git.savannah.gnu.org/cgit/guix.git/commit/?id=4955589f2f343e1862dfae7831d1fc548811d59b I think this makes sense if a branch is a set of patches that are somewhat ready at the point they're pushed as a branch, but obviously has downsides if there's lots of time taken to make changes once it's at the head of the queue (or slow builds, which is more like what's been happening recently). > Our documentation [0] includes both of the following statements: > > "Once a branch is at the front of the queue, wait until sufficient > time has passed for the build farms to have processed the changes, and > for the necessary testing to have happened." > > "Sometimes, a branch may be a work in progress, for example for larger > efforts such as updating the GNOME desktop. In these cases, the branch > name should reflect this by having the ‘wip-’ prefix. The QA > infrastructure will avoid building work-in-progress branches, so that > the available resources can be better focused on building the branches > that are ready to be merged. When the branch is no longer a work in > progress, it should be renamed, with the ‘wip-’ prefix removed, and > only then should the merge requests be created, as documented > earlier." > > [0] > https://guix.gnu.org/manual/devel/en/html_node/Managing-Patches-and-Branches.html > > It seems that we are at the point where the queue has grown so long > (more than "2 months" for the python-team) that teams do not want to > lose their position in the queue. > > Also, mass rebuilds require the build farm, both due to the number of > builds and that builds are not always possible on user hardware (even > for x86-64). > > Is only the branch at the head of the queue being built? If not, why > not allow other branches in a ready state to merge? And if one branch > does block the queue then perhaps we can create a second fast-track > queue for branches not performing world rebuilds. > > What is the distinction between the "qa" service and "ci"? Does c++ > need a specification on ci.guix.gnu.org? On a practical level, QA builds branches using the bordeaux build farm according to the open issues in the queue. CI builds branches as set out in it's specifications. QA used to try and build the top 2 or 3 branches, but I changed this to 1 [2] and limited data.qa.guix.gnu.org to just see the top 2 branches, in an attempt to better focus it's limited resources on the top of the queue. 2: https://git.savannah.gnu.org/cgit/guix/qa-frontpage.git/tree/guix-qa-frontpage/manage-builds.scm#n438 In contrast, CI builds all the specifications all the time and is better able to do so as it's processing of revisions is more parallel compared to the data service, and it has a lot more build machines (at least x86 ones). To directly address your question about the c++-team branch and CI, ideally it would have a specification there since that'll allow CI to build things from that branch. I don't believe there's a process to make this happen, but several people have access so I'd suggest to just ask in the issue/on IRC/on mailing lists until someone creates the specification. > Now, the c++-team branch is no longer in a ready state because I > discovered additional work (ctest has no concept of a "test-target" > and many packages were silently skipping tests). I am confident that I > will have this fixed shortly, certainly within the time that the queue > clears, but I see that I am also contributing to the collective > problem. > > Also, the longer team branches sit in the queue, the more time to > accrue updates, the more we start to look like the old days of the > staging and core-updates branches. I think improvements to managing branches fall in to three areas: - Process - Tooling - Scoping/packaging approach On process, there's already discussion happening about the ordering of the merge queue, that's how python-team ended up next since the core-packages-team issue was closed. I'm unsure about adding a "fast-track" queue as you suggest, but I think it might be helpful to have more discussion about the optimal order. Take elogind-updates, it's behind python-team but seems to have a far tighter scope, maybe it should have gone first? On tooling, as I say, QA used to attempt to build the first 2 or 3 branches. With the data service and build coordination being a bit faster now, I think building the first two branches might be feasible. That might compliment my previous point about optimising the order in which branches are merged, since we'll have more information on a number of branches, not just the one at the top of the queue. Then, I'm also hopeful that we can potentially tweak the packages themselves to make managing changes easier. Often this can look like changing an input to a minimal or pinned variant, thus reducing the interdependency between specific sets of packages. To put this in practical terms, I'm not sure why the scope of python-team seems almost equivalent to core-packages-team. Maybe there's a good reason, but I think we could do with better tooling to more easily see what packages are involved in effecting the most change. If the scope of branches can be smaller, then that should hopefully make it faster to build and push the changes to master. I think there's also several antipatterns to avoid here. I'm still not sold on the teams thing, I know people have been naming branches after teams and I'm fine with that, I'm just treating it as a name and ignoring any association and the docs do similarly. As I say above I think there's a need for more discussion and coordination to get changes to master, if teams push people to only look at branches relevant to the teams they're in or discourage them from looking at or working on branches "outside" of those teams, then that's bad in my view. And while I think that making informed decisions about when to push changes to master is the end goal, I think we do need to be careful about knowingly breaking things. Take the current python-team as an example, the substitute availability looks good, but QA does suggest that some packages are broken (that is they build on master, but not on python-team). There are reasons why breakages shouldn't block pushing to master (like packages being unmaintained upstream), but packages breaking on master is generally discouraging and inconvenient to users and contributors, and something to try and avoid, even if it slows down making other changes. Unfortunately most of the documentation tweaking I was doing last year was difficult to put in to practice since QA was super fragile and wasn't working that well generally. It's still super fragile, but right now it seems to be working better for branches at least. Hopefully we might be able to better iterate the process this year.
signature.asc
Description: PGP signature
