On Mon, Jul 7, 2025 at 11:55 PM Soumyadeep Ghosh < soumyadeepghosh2...@zohomail.in> wrote:
> Hi Ben, > Hi Soumyadeep, > > This is an awesome news! I just wanted to know what would be the problem > in publishing those snaps built inside the new VM based CI. > There is currently no Notary built that supports handling the publishing process at this time, which is why we're unable to publish. > > Thanks, > Soumyadeep Ghosh > Cheers, Ben > > > > > ---- On Mon, 07 Jul 2025 17:19:14 +0530 * bcooks...@kde.org > <bcooks...@kde.org> * wrote ---- > > Hi all, > > I'm happy to announce that VM based CI is now in a state where it is ready > to begin making the transition into general availability. As part of this > i'm happy to advise that builds for Snaps (but not their publishing) are > also becoming generally available, and FreeBSD will be updating to Qt 6.9 > as well. > > This transition will also mark the general retirement of Docker based CI, > with Docker functionality only being retained for website builds and linter > jobs (such as xml-lint, cppcheck, etc). > > As part of this support for Qt 5 (except for general Linux) is also being > retired. This includes all FreeBSD and Windows support, as well as all > binary support (Appimages, Windows exes, etc) > > *Steps you need to take:* > > If you only inherit the templates from sysadmin/ci-utilities, and do not > have custom jobs, then no action is needed on your part. > > If you have custom jobs which are based on the templates currently in > sysadmin/ci-utilities then you should examine the diff in > https://invent.kde.org/sysadmin/ci-utilities/-/merge_requests/507 to > ensure that no changes are needed to your job. > > Custom jobs not based on the sysadmin provided templates likely do not > need to be changed. > > Projects that are Qt 5 based should remove includes for everything except > Linux general (linux.yml) as those templates will be removed as part of > this roll out. > > *Timeline for the transition:* > > Over the next week the first node (node6.ci.kde.org) will be taken out of > rotation for running existing container based jobs to facilitate testing of > the automated deployment. Once this has been validated, the conversion of > the remaining nodes (node2, node3, node4, node5) will be scheduled - > converting them all to be VM based CI runners. > > To allow for final legacy jobs to be completed, node1 will be left to run > remaining legacy jobs for a period of time (to be determined). Once that > finishes up, that node will also be converted. > > As part of this the dedicated VM runners currently supplied for building > KDE Linux and Snaps will also be retired. Support for Yocto builds is out > of scope for this transition at this time but that may be re-evaluated in > the future. > > *What specs will the VMs be provided with?* > > By default builds will be provided with 8 vCPU cores, 16GB RAM and 200GB > of disk space, although some of that disk space will be occupied by the VM > OS, development tools, etc. > Builds requiring additional resources should file a Sysadmin ticket. > > VMs will have a shared mount provided from the host at /mnt that allows > for artifacts to be cached between builds on that node. This is mainly > intended to be used for caching dependencies and other downloaded artifacts > and should not be relied on to transfer results between jobs (as there is > no guarantee contents won't go away, or that jobs will be allocated to the > same build node). > > *What benefits will we get as part of this change?* > > For Linux based builds, as these are now running in a fully fledged > system, anything that depends on system level services (like CUPS or > systemd/logind) or which needs to undertake actions normally restricted in > a Docker/Podman container (like manipulating FUSE mounts or interacting > with hardware like a TPM) should now be able to be unit tested. > > For FreeBSD based builds, these images no longer have to be built manually > by hand, and are now easily available to be run locally on developer > systems for local testing. Reliability of FreeBSD builds should also > improve with the elimination of the dependency on the Podman daemon being > running on the build node. > > For Windows based builds, these should run faster due to the elimination > of the overhead of the Docker overlay file system, which on Windows > introduces significant overhead to IO operations. For Kate, regular Windows > CI build times were reduced from 6 minutes (under Docker) to 3 minutes 40 > seconds (in a VM based CI setup). This includes the overhead of > provisioning the VM and waiting for Windows to start. We should also see > improvements to builder availability, as the out of memory issues that have > caused Windows build nodes to be killed by the OOM-killer are not possible > in a VM based CI environment. > > For binary builds, Linux support is enhanced by it now being possible to > build Snaps, as well as Flatpaks no longer requiring workarounds to be > built, while Appimages are now able to be run if needed. These are actions > that have all previously been restricted by operating in a container > environment. > > The system will also generally benefit from being able to scale build > capacity for different OSes on an as needed basis within our available > hardware (so long build queues for Windows post major release events should > become less of an issue). > > As an additional benefit, the system will require significantly less work > to maintain. Currently each build node, along with the FreeBSD and Windows > VM thereon, have to be maintained by hand and disk space allocated between > them in a fixed fashion. This means that any cleanup from stale disk > images, over-filled caches, etc. has to be done 3 times on each build node > (being the Linux host as well as the FreeBSD and Windows guest VMs). > Currently provisioning new nodes is significantly labour intensive as well > (see > https://invent.kde.org/sysadmin/ci-utilities/-/blob/master/gitlab-templates/README.md > for the instructions). > > This is essentially completely eliminated with the transition to VM based > CI, with the majority of the deployment now being possible using Ansible > with the only manual step being the registration with Gitlab - which is a > fairly quick process taking less than 20 minutes per node. Maintenance is > significantly reduced as each node only needs one set of cleanup - not > three. > > Should there be any questions on the above please let me know. > > Many thanks, > Ben > > >
