The QEMU pipeline has as many as 120+ jobs, almost all of which are always run. This is wasteful in many ways. When merging to the main repo, Peter runs CI on 'staging' branch, which then gets immediately re-run on the 'master' branch. The first patch in this series culls that redundancy by always skipping CI on pushes to 'master'.
I suggest we setup a nightly scheduled job that runs on current contents of 'master' once a day, as a way to catch bitrot from OS distro's changing under us. I think we want the first patch as a high priority, since I think the duplicated jobs on master are really impacting our integration with Cirrus CI, delaying jobs since Cirrus has very strictly applied concurrency limits. The second patch is more speculative, and illustrates how to we can reduce the number of CI jobs run for *all* contributors, when they push to gitlab. There is a key decision to make here between: a. run no jobs at all by default on push b. run a few jobs by default on push to catch 90% of problems c. run all jobs on by default on push Currently we're doing (c), and patch 2 implements (b), with an easy opt-in to trigger (c). There is a credible variant of patch 2 though, which could easily implement (a) with an opt-in to run (b), and a further opt-in to run (c). Or we can get more advanced and add ability for individual contributors to decide between (a) and (b) for default behaviour, based on a variable configured globally in their gitlab fork. There is an illustration of results of (b) here: https://gitlab.com/berrange/qemu/-/pipelines/339281325 Note that for all the non-defualt jobs, I've left them as manually triggerable. This makes a the pipeline UI a bit cluttered by default but lets you trigger jobs from the web UI individually. An alternative would be to never create the non-defualt jobs at all. They would have to be requested explicitly at time of push in some way. Daniel P. Berrangé (2): gitlab: prevent CI jobs running pushes to default branch or tags gitlab: cut down on jobs run by default in user forks .gitlab-ci.d/buildtest-template.yml | 17 +--- .gitlab-ci.d/buildtest.yml | 14 +++ .gitlab-ci.d/cirrus.yml | 5 +- .gitlab-ci.d/container-core.yml | 3 + .gitlab-ci.d/container-cross.yml | 9 +- .gitlab-ci.d/container-template.yml | 1 + .gitlab-ci.d/containers.yml | 2 + .gitlab-ci.d/crossbuild-template.yml | 3 + .gitlab-ci.d/crossbuilds.yml | 3 + .gitlab-ci.d/qemu-project.yml | 1 + .gitlab-ci.d/workflow.yml | 146 +++++++++++++++++++++++++++ .gitlab-ci.yml | 21 +--- 12 files changed, 188 insertions(+), 37 deletions(-) create mode 100644 .gitlab-ci.d/workflow.yml -- 2.31.1
