On Thu, Jun 25, 2020 at 12:26:53PM +0100, Daniel P. Berrangé wrote: > On Thu, Jun 25, 2020 at 01:15:52PM +0200, Thomas Huth wrote: > > On 22/06/2020 17.33, Daniel P. Berrangé wrote: > > > The current gitlab CI jobs are quite inefficient because they > > > use the generic distro images and then apt-get/dnf install > > > extra packages every time. > > > > > > The other downside is that the container environment used is > > > only defined in thte .gitlab-ci.yml file, so it tedious to > > > reproduce locally. > > > > > > We already have containers defined in tests/docker for use by > > > developers building locally. We can use these for CI systems > > > too if we just had a way to build them.... > > > > > > ...GitLab CI offers such a way. We can use docker-in-docker > > > to build the images at the start of the CI cycle, and use > > > the built images in later jobs. > > > > > > These later jobs are now faster because they're not having > > > to install any software. > > > > Did you see any speed-up? I had a look at some pipelines, and it seems to me > > that they rather got slower now? For example, this is the system1 pipeline > > before your change: > > > > https://gitlab.com/huth/qemu/-/jobs/610924897 > > > > and after your change: > > > > https://gitlab.com/huth/qemu/-/jobs/611069374 > > > > Duration went up from 35 minutes to 42 minutes. > > > > Seems also to happen in your builds, before the change: > > > > https://gitlab.com/berrange/qemu/-/jobs/582995084 > > > > and after the change: > > > > https://gitlab.com/berrange/qemu/-/jobs/606175927 > > > > ... went from 36 minutes up to 42 minutes. > > > > Could be a coincidence due to the load on the shared runners, but it looks > > at least a little bit suspicious... > > I think the difference is because we're building more features now. The > dockerfiles have provided more build pre-requisites that the old gitlab > recipe did. > > If you compare the configure summary, I see the new build now covers > SDL, curses, curl, pulseaudio, virtiofs, SASL, libjpeg, xen, docs > and a few more. So we've saved time by not intsallling many packages > each time, but consumed a greater amount of time by compiling more > features.
Oh a missed a lot more actually - there's also spice, opengl, libiscsi, libnfs, libusb, seccomp, libssh, lzo, snappy, bzip, zstd, numa and udev too. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
