On Thu, Dec 09, 2021 at 11:31:24AM +0100, Thomas Huth wrote: > Cirrus-CI provides KVM in their Linux containers, so we can also run > our VM-based NetBSD and OpenBSD build jobs there. > Since the VM installation might take a while, we only run the "help" > target on the first invocation to avoid timeouts, and then only check > the build during the next run, once the base image has been cached. > For the the build tests, we also only use very a limited set of target > CPUs since compiling in these VMs is not very fast (especially the > build on OpenBSD seems to be incredibly slow). > For the time being, the jobs are also marked as manually only, since > this double-indirect setup (with the cirrus-run script and VMs in > the Cirrus-CI containers) might fail more often than the other jobs.
I think they'll have to be manual forever basically unless something changes in cirrus. Historically we've had trouble with the cirrus jobs timing out. This was ultimately a result of the fact that only 2 cirrus jobs can run concurrently, and we had duplicate jobs being scheduled on 'master' and 'staging'. This resulted in 4 jobs being queued and most of the time, and because each job took > 30 minutes, two of them would frequently hit the gitlab job 1 hour timeout. Unless we can ensure that /all/ our cirrus jobs will reliably completed in about 20 minutes in normal case (30 mins if cirrus is being slow), then we can't have more than 2 cirrus jobs as one or more will end up going over the 1 hour cutoff. The idea of having NetBSD/OpenBSD jobs is good, but I think it feels like a case where we're going to need to look at using custom runners if we want them triggere on 'staging'. Manual jobs could be ok for contributors forks at most. > Signed-off-by: Thomas Huth <th...@redhat.com> > --- > If you're interested, you can see some runs here: > NetBSD: https://gitlab.com/thuth/qemu/-/jobs/1865965079 > OpenBSD: https://gitlab.com/thuth/qemu/-/jobs/1866116634 > > .gitlab-ci.d/cirrus.yml | 35 +++++++++++++++++++++++++++++++ > .gitlab-ci.d/cirrus/kvm-build.yml | 31 +++++++++++++++++++++++++++ > 2 files changed, 66 insertions(+) > create mode 100644 .gitlab-ci.d/cirrus/kvm-build.yml > > diff --git a/.gitlab-ci.d/cirrus.yml b/.gitlab-ci.d/cirrus.yml > index 90bbe9f8ff..f2d4c0ca4f 100644 > --- a/.gitlab-ci.d/cirrus.yml > +++ b/.gitlab-ci.d/cirrus.yml > @@ -64,3 +64,38 @@ x64-macos-11-base-build: > PATH_EXTRA: /usr/local/opt/ccache/libexec:/usr/local/opt/gettext/bin > PKG_CONFIG_PATH: > /usr/local/opt/curl/lib/pkgconfig:/usr/local/opt/ncurses/lib/pkgconfig:/usr/local/opt/readline/lib/pkgconfig > TEST_TARGETS: check-unit check-block check-qapi-schema check-softfloat > check-qtest-x86_64 > + > + > +# The following jobs run VM-based tests via KVM on a Linux-based Cirrus-CI > job > +.cirrus_kvm_job: > + stage: build > + image: registry.gitlab.com/libvirt/libvirt-ci/cirrus-run:master > + needs: [] > + timeout: 80m > + allow_failure: true > + script: > + - sed -e "s|[@]CI_REPOSITORY_URL@|$CI_REPOSITORY_URL|g" > + -e "s|[@]CI_COMMIT_REF_NAME@|$CI_COMMIT_REF_NAME|g" > + -e "s|[@]CI_COMMIT_SHA@|$CI_COMMIT_SHA|g" > + -e "s|[@]NAME@|$NAME|g" > + -e "s|[@]CONFIGURE_ARGS@|$CONFIGURE_ARGS|g" > + -e "s|[@]TEST_TARGETS@|$TEST_TARGETS|g" > + <.gitlab-ci.d/cirrus/kvm-build.yml >.gitlab-ci.d/cirrus/$NAME.yml > + - cat .gitlab-ci.d/cirrus/$NAME.yml > + - cirrus-run -v --show-build-log always .gitlab-ci.d/cirrus/$NAME.yml > + rules: > + - when: manual > + > +x86-netbsd: > + extends: .cirrus_kvm_job > + variables: > + NAME: netbsd > + CONFIGURE_ARGS: > --target-list=x86_64-softmmu,ppc64-softmmu,aarch64-softmmu > + TEST_TARGETS: check > + > +x86-openbsd: > + extends: .cirrus_kvm_job > + variables: > + NAME: openbsd > + CONFIGURE_ARGS: --target-list=i386-softmmu,riscv64-softmmu,mips64-softmmu > + TEST_TARGETS: check > diff --git a/.gitlab-ci.d/cirrus/kvm-build.yml > b/.gitlab-ci.d/cirrus/kvm-build.yml > new file mode 100644 > index 0000000000..4334fabf39 > --- /dev/null > +++ b/.gitlab-ci.d/cirrus/kvm-build.yml > @@ -0,0 +1,31 @@ > +container: > + image: fedora:35 > + cpu: 4 > + memory: 8Gb > + kvm: true > + > +env: > + CIRRUS_CLONE_DEPTH: 1 > + CI_REPOSITORY_URL: "@CI_REPOSITORY_URL@" > + CI_COMMIT_REF_NAME: "@CI_COMMIT_REF_NAME@" > + CI_COMMIT_SHA: "@CI_COMMIT_SHA@" > + > +@NAME@_task: > + @NAME@_vm_cache: > + folder: $HOME/.cache/qemu-vm > + install_script: > + - dnf update -y > + - dnf install -y git make openssh-clients qemu-img qemu-system-x86 wget > + clone_script: > + - git clone --depth 100 "$CI_REPOSITORY_URL" . > + - git fetch origin "$CI_COMMIT_REF_NAME" > + - git reset --hard "$CI_COMMIT_SHA" > + build_script: > + - if [ -f $HOME/.cache/qemu-vm/images/@NAME@.img ]; then > + make vm-build-@NAME@ J=$(getconf _NPROCESSORS_ONLN) > + EXTRA_CONFIGURE_OPTS="@CONFIGURE_ARGS@" > + BUILD_TARGET="@TEST_TARGETS@" ; > + else > + make vm-build-@NAME@ J=$(getconf _NPROCESSORS_ONLN) BUILD_TARGET=help > + EXTRA_CONFIGURE_OPTS="--disable-system --disable-user > --disable-tools" ; > + fi > -- > 2.27.0 > > Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
