Alex Bennée <alex.ben...@linaro.org> writes:
> We were using quite and old runner on our machines and running into
> issues with stalling jobs. Gitlab in the meantime now reliably provide
> the latest packaged versions of the runner under a stable URL. This
> update:
>
> - creates a per-arch subdir for builds
> - switches from binary tarballs to deb packages
> - re-uses the same binary for the secondary runner
> - updates distro check for second to 22.04
>
> So far I've tested on aarch64.ci.qemu.org but I shall do s390x next as
> its having issues with stale runners as well.
>
> Signed-off-by: Alex Bennée <alex.ben...@linaro.org>
> ---
> scripts/ci/setup/gitlab-runner.yml | 42 ++++++++++++++++--------------
> scripts/ci/setup/vars.yml.template | 2 --
> 2 files changed, 22 insertions(+), 22 deletions(-)
>
> diff --git a/scripts/ci/setup/gitlab-runner.yml
> b/scripts/ci/setup/gitlab-runner.yml
> index 33128be85d..05cdb2ae4b 100644
> --- a/scripts/ci/setup/gitlab-runner.yml
> +++ b/scripts/ci/setup/gitlab-runner.yml
> @@ -30,6 +30,11 @@
> home: /home/gitlab-runner
> shell: /bin/bash
>
> + - name: Create working directories for build
> + file:
> + path: "/home/gitlab-runner/{{ ansible_facts[\"architecture\"] }}"
> + state: directory
> +
> - name: Remove the .bash_logout file when on Ubuntu systems
> file:
> path: /home/gitlab-runner/.bash_logout
> @@ -50,17 +55,17 @@
>
> - name: Download the matching gitlab-runner
> get_url:
> - dest: /usr/local/bin/gitlab-runner
> - url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{
> gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-{{
> gitlab_runner_arch }}"
> - owner: gitlab-runner
> - group: gitlab-runner
> - mode: u=rwx,g=rwx,o=rx
> + dest: "/root/"
> + url:
> "https://gitlab-runner-downloads.s3.amazonaws.com/latest/deb/gitlab-runner_{{
> gitlab_runner_arch }}.deb"
> +
> + - name: Install gitlab-runner via package manager
> + apt: deb="/root/gitlab-runner_{{ gitlab_runner_arch }}.deb"
>
> - name: Register the gitlab-runner
> - command: "/usr/local/bin/gitlab-runner register --non-interactive
> --url {{ gitlab_runner_server_url }} --registration-token {{
> gitlab_runner_registration_token }} --executor shell --tag-list {{
> ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower
> }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{
> ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"]
> }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"]
> }})'"
> + command: "/usr/bin/gitlab-runner register --non-interactive --url {{
> gitlab_runner_server_url }} --registration-token {{
> gitlab_runner_registration_token }} --executor shell --tag-list {{
> ansible_facts[\"architecture\"] }},{{ ansible_facts[\"distribution\"]|lower
> }}_{{ ansible_facts[\"distribution_version\"] }} --description '{{
> ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"]
> }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"]
> }})'"
>
> - name: Install the gitlab-runner service using its own functionality
> - command: /usr/local/bin/gitlab-runner install --user gitlab-runner
> --working-directory /home/gitlab-runner
> + command: "/usr/bin/gitlab-runner install --user gitlab-runner
> --working-directory /home/gitlab-runner/{{ ansible_facts[\"architecture\"] }}"
> register: gitlab_runner_install_service_result
> failed_when: "gitlab_runner_install_service_result.rc != 0 and
> \"already exists\" not in gitlab_runner_install_service_result.stderr"
>
> @@ -70,33 +75,30 @@
> state: started
> enabled: yes
>
> - - name: Download secondary gitlab-runner
> - get_url:
> - dest: /usr/local/bin/gitlab-runner-arm
> - url: "https://s3.amazonaws.com/gitlab-runner-downloads/v{{
> gitlab_runner_version }}/binaries/gitlab-runner-{{ gitlab_runner_os }}-arm"
> - owner: gitlab-runner
> - group: gitlab-runner
> - mode: u=rwx,g=rwx,o=rx
> + - name: Create working directories secondary runner
> + file:
> + path: "/home/gitlab-runner/arm"
> + state: directory
> when:
> - ansible_facts['distribution'] == 'Ubuntu'
> - ansible_facts['architecture'] == 'aarch64'
> - - ansible_facts['distribution_version'] == '20.04'
> + - ansible_facts['distribution_version'] == '22.04'
>
> - name: Register secondary gitlab-runner
> - command: "/usr/local/bin/gitlab-runner-arm register --non-interactive
> --url {{ gitlab_runner_server_url }} --registration-token {{
> gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{
> ansible_facts[\"distribution\"]|lower }}_{{
> ansible_facts[\"distribution_version\"] }} --description '{{
> ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"]
> }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"]
> }})'"
> + command: "/usr/bin/gitlab-runner register --non-interactive --url {{
> gitlab_runner_server_url }} --registration-token {{
> gitlab_runner_registration_token }} --executor shell --tag-list aarch32,{{
> ansible_facts[\"distribution\"]|lower }}_{{
> ansible_facts[\"distribution_version\"] }} --description '{{
> ansible_facts[\"distribution\"] }} {{ ansible_facts[\"distribution_version\"]
> }} {{ ansible_facts[\"architecture\"] }} ({{ ansible_facts[\"os_family\"]
> }})'"
> when:
> - ansible_facts['distribution'] == 'Ubuntu'
> - ansible_facts['architecture'] == 'aarch64'
> - - ansible_facts['distribution_version'] == '20.04'
> + - ansible_facts['distribution_version'] == '22.04'
I think everything bellow here can be dropped now as both runners seem
to be showing the same runner ids:
13:42:48 [root@aarch64:/e/systemd] + systemctl status gitlab-runner-arm
● gitlab-runner-arm.service - GitLab Runner
Loaded: loaded (/etc/systemd/system/gitlab-runner-arm.service; enabled;
vendor preset: enabled)
Active: active (running) since Fri 2023-01-06 13:42:45 UTC; 9s ago
Main PID: 38362 (gitlab-runner)
Tasks: 33 (limit: 307985)
Memory: 16.0M
CPU: 81ms
CGroup: /system.slice/gitlab-runner-arm.service
└─38362 /usr/bin/gitlab-runner run --working-directory
/home/gitlab-runner/arm --config /etc/gitlab-runner/config.toml --service
gitlab-runner-arm --user gitla>
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38362]: Running in
system-mode.
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38362]:
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38362]: Configuration
loaded builds=0
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38362]: listen_address not
defined, metrics & debug endpoints disabled builds=0
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38362]:
[session_server].listen_address not defined, session endpoints disabled
builds=0
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38362]: Initializing
executor providers builds=0
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38362]: Checking for
jobs...nothing runner=HXc1tXPb
Jan 06 13:42:46 aarch64.ci.qemu.org gitlab-runner[38362]: Checking for
jobs...nothing runner=pwrzaBKo
Jan 06 13:42:47 aarch64.ci.qemu.org gitlab-runner[38362]: Checking for
jobs...nothing runner=E8PpwMky
Jan 06 13:42:47 aarch64.ci.qemu.org gitlab-runner[38362]: Checking for
jobs...nothing runner=syqX1xoo
13:43:06 [root@aarch64:/e/systemd] + systemctl status gitlab-runner
● gitlab-runner.service - GitLab Runner
Loaded: loaded (/etc/systemd/system/gitlab-runner.service; enabled;
vendor preset: enabled)
Active: active (running) since Fri 2023-01-06 13:42:42 UTC; 28s ago
Main PID: 38270 (gitlab-runner)
Tasks: 27 (limit: 307985)
Memory: 14.3M
CPU: 70ms
CGroup: /system.slice/gitlab-runner.service
└─38270 /usr/bin/gitlab-runner run --working-directory
/home/gitlab-runner --config /etc/gitlab-runner/config.toml --service
gitlab-runner --user gitlab-runner
Jan 06 13:42:42 aarch64.ci.qemu.org gitlab-runner[38270]: Running in
system-mode.
Jan 06 13:42:42 aarch64.ci.qemu.org gitlab-runner[38270]:
Jan 06 13:42:42 aarch64.ci.qemu.org gitlab-runner[38270]: Configuration
loaded builds=0
Jan 06 13:42:42 aarch64.ci.qemu.org gitlab-runner[38270]: listen_address not
defined, metrics & debug endpoints disabled builds=0
Jan 06 13:42:42 aarch64.ci.qemu.org gitlab-runner[38270]:
[session_server].listen_address not defined, session endpoints disabled
builds=0
Jan 06 13:42:42 aarch64.ci.qemu.org gitlab-runner[38270]: Initializing
executor providers builds=0
Jan 06 13:42:43 aarch64.ci.qemu.org gitlab-runner[38270]: Checking for
jobs...nothing runner=HXc1tXPb
Jan 06 13:42:43 aarch64.ci.qemu.org gitlab-runner[38270]: Checking for
jobs...nothing runner=pwrzaBKo
Jan 06 13:42:44 aarch64.ci.qemu.org gitlab-runner[38270]: Checking for
jobs...nothing runner=E8PpwMky
Jan 06 13:42:45 aarch64.ci.qemu.org gitlab-runner[38270]: Checking for
jobs...nothing runner=syqX1xoo
--
Alex Bennée
Virtualisation Tech Lead @ Linaro
