Re: [PATCH] gitlab-ci: Replace Docker with Kaniko

On Fri, May 17, 2024 at 9:14 AM Daniel P. Berrangé wrote: > > On Thu, May 16, 2024 at 07:24:04PM +0100, Daniel P. Berrangé wrote: > > On Thu, May 16, 2024 at 05:52:43PM +0100, Camilla Conte wrote: > > > Enables caching from the qemu-project repository. > > > >

[PATCH] gitlab-ci: Replace Docker with Kaniko

62 Previous attempts, for the records: - Alex Bennée: https://lore.kernel.org/qemu-devel/20230330101141.30199-12-alex.ben...@linaro.org/ - Camilla Conte (me): https://lore.kernel.org/qemu-devel/20230531150824.32349-6-cco...@redhat.com/ Signed-off-by: Camilla Conte --- .gitlab-ci

Re: [PATCH] gitlab: switch from 'stable' to 'latest' docker container tags

e9d51764e..b4d7eef688 100644 > --- a/.gitlab-ci.d/opensbi.yml > +++ b/.gitlab-ci.d/opensbi.yml > @@ -42,9 +42,9 @@ > docker-opensbi: >extends: .opensbi_job_rules >stage: containers > - image: docker:stable > + image: docker:latest >services: > -- docker:stable-dind > +- docker:dind >variables: > GIT_DEPTH: 3 > IMAGE_TAG: $CI_REGISTRY_IMAGE:opensbi-cross-build > -- > 2.40.1 > Reviewed-by: Camilla Conte

[PATCH 5/5] gitlab-ci: Replace Docker with Kaniko

Dockerfile is using COPY or ADD instructions. Does not enable reproducible builds as that results in builds failing with an out of memory error. See issue "Using --reproducible loads entire image into memory": https://github.com/GoogleContainerTools/kaniko/issues/862 Signed-off-by: Cam

[PATCH 1/5] gitlab-ci: Remove unused Python package

Python should have been removed in this commit: https://gitlab.com/qemu-project/qemu/-/commit/94b8b146df84ba472f461398d93fb9cdf0db8f94 Signed-off-by: Camilla Conte --- .gitlab-ci.d/container-template.yml | 1 - 1 file changed, 1 deletion(-) diff --git a/.gitlab-ci.d/container-template.yml b

Improvements to Gitlab CI container builds

Given my recent deep-dive in this part of the Gitlab CI, I figured I could propose some improvements that came to mind along the way. The last patch removes the need for Docker in Docker.

[PATCH 2/5] gitlab-ci: Reference rules instead of extends

This allows for the jobs to extend different templates while having a shared template for rules. Docs: https://docs.gitlab.com/ee/ci/jobs/job_control.html#reuse-rules-in-different-jobs Signed-off-by: Camilla Conte --- .gitlab-ci.d/opensbi.yml | 6 -- 1 file changed, 4 insertions(+), 2

[PATCH 3/5] gitlab-ci: Move variables outside script

Also split long line for clarity. Docs: https://docs.gitlab.com/ee/ci/variables/#use-cicd-variables-in-other-variables Signed-off-by: Camilla Conte --- .gitlab-ci.d/container-template.yml | 17 - 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/.gitlab-ci.d

[PATCH 4/5] gitlab-ci: Use container template for opensbi

mple of the old URLs: - registry.gitlab.com/qemu-project/qemu:00a0bdc... - registry.gitlab.com/qemu-project/qemu:opensbi-cross-build Example of the new URL: - registry.gitlab.com/qemu-project/qemu/qemu/opensbi-cross-build Signed-off-by: Camilla Conte --- .gitlab-ci.d/opensbi.yml | 23 --

Re: Add CI configuration for Kubernetes

On Tue, May 23, 2023 at 2:08 PM Richard Henderson wrote: > > On 5/23/23 01:24, Camilla Conte wrote: > > On Mon, May 22, 2023 at 11:52 PM Richard Henderson > > wrote: > >> This does not work: > >> > >> https://gitlab.com/qemu-project/qemu/-/pipeli

Re: [PATCH v2 3/5] Add loop over docker info

On Tue, May 23, 2023 at 2:06 PM Richard Henderson wrote: > > On 5/22/23 10:41, Camilla Conte wrote: > > +- until docker info; do sleep 1; done > > How long are you expecting this to have to wait? > Less than 5 seconds? I remember seeing jobs where it took longer tha

[PATCH v3 5/5] Add Kubernetes runner configuration

Custom values for the gitlab-runner Helm chart. See https://wiki.qemu.org/Testing/CI/KubernetesRunners. Signed-off-by: Camilla Conte --- Changes since previous version (PATCH v2 5/5): - Update ubuntu tag to 22.04. I tested it and it works. - Remove concurrent jobs limits (use defaults). .../ci

Re: Add CI configuration for Kubernetes

On Mon, May 22, 2023 at 11:52 PM Richard Henderson wrote: > This does not work: > > https://gitlab.com/qemu-project/qemu/-/pipelines/875254290 My bad, sorry. I didn't update the runner with the new values.yaml. The Docker environment variables were missing so it's looking for the Docker server at

[PATCH v2 3/5] Add loop over docker info

Wait for docker info to return successfuly to ensure that the docker server (daemon) started. This is needed for jobs running on Kubernetes. See https://wiki.qemu.org/Testing/CI/KubernetesRunners. Signed-off-by: Camilla Conte --- .gitlab-ci.d/container-template.yml | 2 +- .gitlab-ci.d

[PATCH v2 5/5] Add Kubernetes runner configuration

Custom values for the gitlab-runner Helm chart. See https://wiki.qemu.org/Testing/CI/KubernetesRunners. Signed-off-by: Camilla Conte --- .../ci/gitlab-kubernetes-runners/values.yaml | 30 +++ 1 file changed, 30 insertions(+) create mode 100644 scripts/ci/gitlab-kubernetes

Add CI configuration for Kubernetes

Here's a second version (v2) of patches to support the Kubernetes runner for Gitlab CI. You can find the v1 thread here: https://lore.kernel.org/qemu-devel/20230407145252.32955-1-cco...@redhat.com/.

[PATCH v2 2/5] Use docker "stable" tag

Use the same tag in all jobs. Signed-off-by: Camilla Conte --- .gitlab-ci.d/container-template.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.gitlab-ci.d/container-template.yml b/.gitlab-ci.d/container-template.yml index 519b8a9482..11569dd900 100644 --- a/.gitlab

[PATCH v2 4/5] Add CI variable RUNNER_TAG

This allows to set a job tag dinamically. We need this to be able to select the Kubernetes runner. See https://wiki.qemu.org/Testing/CI/KubernetesRunners. Signed-off-by: Camilla Conte --- .gitlab-ci.d/qemu-project.yml | 7 +++ 1 file changed, 7 insertions(+) diff --git a/.gitlab-ci.d/qemu

[PATCH v2 1/5] Remove redundant CI variables

These are not needed when using gitlab.com shared runners. Signed-off-by: Camilla Conte --- .gitlab-ci.d/opensbi.yml | 3 --- 1 file changed, 3 deletions(-) diff --git a/.gitlab-ci.d/opensbi.yml b/.gitlab-ci.d/opensbi.yml index 9a651465d8..13070575b6 100644 --- a/.gitlab-ci.d/opensbi.yml +++ b

Re: [PATCH 1/2] Add CI configuration for Kubernetes

On Fri, May 19, 2023 at 1:51 PM Daniel P. Berrangé wrote: > > On Fri, May 19, 2023 at 01:33:50PM +0100, Camilla Conte wrote: > > On Fri, May 19, 2023 at 10:00 AM Daniel P. Berrangé > > wrote: > > > > > > On Fri, Apr 07, 2023 at 03:52:51PM +0100, Camilla Cont

Re: [PATCH 1/2] Add CI configuration for Kubernetes

On Fri, May 19, 2023 at 10:00 AM Daniel P. Berrangé wrote: > > On Fri, Apr 07, 2023 at 03:52:51PM +0100, Camilla Conte wrote: > > Configure Gitlab CI to run on Kubernetes > > according to the official documentation. > > https://docs.gitlab.com/ee/ci/docker/using_docker_bui

Re: gitlab shared runner time expired

Hi! On Fri, May 19, 2023 at 9:35 AM Paolo Bonzini wrote: > You should be able to run pipelines almost as usual, just with > > git push -o ci.variable=RUNNER_TAG=k8s ... > > Camilla, please confirm that this is correct. Looks good.

[PATCH 1/2] Add CI configuration for Kubernetes

is replaced with Kaniko, these changes can be reverted. I documented what I did to set up the Kubernetes runner on the wiki: https://wiki.qemu.org/Testing/CI/KubernetesRunners Signed-off-by: Camilla Conte --- .gitlab-ci.d/container-template.yml | 6 +++--- .gitlab-ci.d/default.yml

[PATCH 2/2] Raise crash-test-debian timeout to 90 minutes

When running on the Kubernetes runner, this CI job is timing out. Raise the limit to give the job enough time to run. Signed-off-by: Camilla Conte --- .gitlab-ci.d/buildtest.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index