Daniel P. Berrangé <berra...@redhat.com> writes:
> On Tue, Feb 28, 2023 at 02:29:12PM +0100, Philippe Mathieu-Daudé wrote: >> On 28/2/23 13:58, Daniel P. Berrangé wrote: >> > On Tue, Feb 28, 2023 at 12:58:54PM +0100, Philippe Mathieu-Daudé wrote: >> > > On 24/2/23 19:08, Alex Bennée wrote: >> > > > This series attempts to remove our dependence on the docker.py script >> > > > and build things directly with the appropriate tool. I've been >> > > > noodling around with how we build images on gitlab to see if they can >> > > > cache better because the normal case should be we don't need to >> > > > rebuild everything if the upstream distro hasn't updated its package >> > > > list. >> > > > >> > > > Anyway what do people think? >> > > >> > > Removing dind limitation is interesting. >> > > >> > > Unrelated, can we tag registry.gitlab.com/qemu-project's >> > > docker images along with releases? >> > >> > Can you elaborate on this ? >> > >> > We're only using the images for CI purposes and they must always reflect >> > the current state of git master. We're using a fixed docker tag 'latest', >> > as that avoids the container registry growing arbitrarily large. >> > >> > Our CI rules should prevent the pipelines running on stable branches, >> > so we shouldn't need container tags for stable. >> >> I'm not suggesting keeping jobs to build, but doing a snapshot of the >> released containers. >> >> I.e. when we release 8.0, we should tag qemu/fedora:v8.0 and never touch >> it again. This is useful when bisecting pre-v8, but also to build pre-v8 >> and do performance comparison. One shouldn't have to upgrade such >> container (in particular when package mirror disappear), since it >> already contains all we need. > > The main risk with this is the impact on our storage quota. With the > OSS Program membership IIUC we get Ultimate level features which > is 250 GB of storage, across git repos, pipeline cache/artifacts/logs, > container registry. > > Currently they have no way to enforce that since their accounting of > usage is not accurate enough. They're working on fixing that so at > somepoint we'll be subject to the 250 GB limit. > > What I don't know is how much storage we're currently using across > the /qemu-project namespace, and what extra is implied by taking > a snapshot of our container registry 3 times a year. I'm expecting > it to probably be in the high 10's of GB though for the container > registry. Currently we are using: 86.6 Gb of artefacts 28.5 Gb of containers 220 Mb of file uploads 24.8Mb of git repo We could probably cut down a lot of usage of artefacts by either not building full fat ones with debug symbols and passing between layers or tweaking the build system to prevent re-building of object files if the final binary is present in the file system. -- Alex Bennée Virtualisation Tech Lead @ Linaro
