David Marchand <david.march...@redhat.com> writes:
> With the recent changes in terms of free access to the Travis CI, let's
> offer an alternative with GitHub Actions.
> Running jobs on ARM is not supported unless using external runners, so
> this commit only adds builds for x86_64 and cross compiling for i386 and
> aarch64.
>
> Differences with the Travis CI integration:
> - Error logs are not dumped to the console when something goes wrong.
> Instead, they are gathered in a "catch-all" step and attached as
> artifacts.
> - A cache entry is stored once and for all, but if no cache is found you
> can inherit from the default branch cache. The cache is 5GB large, for
> the whole git repository.
> - The maximum retention of logs and artifacts is 3 months.
> - /home/runner is world writable, so a workaround has been added for
> starting dpdk processes.
> - Ilya, working on OVS GHA support, noticed that jobs can run with
> processors that don't have the same capabilities. For DPDK, this
> impacts the ccache content since everything was built with
> -march=native so far, and we will end up with binaries that can't run
> in a later build. The problem has not been seen in Travis CI (?) but
> it is safer to use a fixed "-Dmachine=default" in any case.
That's because the build machine and test machine are the same, but I
think GHA uses a different model, and will spawn a new environment for
the steps. I'm not 100% sure, because it's all supposed to be a
black-box.
> - Scheduling jobs is part of the configuration and takes the form of a
> crontab. A build is scheduled every Monday at 0:00 (UTC) to provide a
> default ccache for the week (useful for the ovsrobot).
>
> Signed-off-by: David Marchand <david.march...@redhat.com>
> ---
Acked-by: Aaron Conole <acon...@redhat.com>
> Changelog since v1:
> - changed shell variables value in CI scripts and Travis configuration
> (s/=[^\$]*1/=\1true), this makes it easier for GHA,
> - forced compilation as 'default' to avoid random unit tests issues in
> GHA,
> - scheduled a run per week on Monday at 0:00 UTC,
> - updated the ccache key:
> - no need to depend on the default-library parameter since this
> parameter only impacts the linking of dpdk binaries,
> - the week when the cache is generated is added so that jobs in
> other branches can benefit from a recent cache (mimicking what we had
> for the robot in Travis),
> - realigned documentation generation with what is done in Travis:
> generating the doc in all jobs was a waste of resources,
>
> ---
> .ci/linux-build.sh | 17 +++---
> .github/workflows/build.yml | 100 ++++++++++++++++++++++++++++++++++++
> .travis.yml | 24 ++++-----
> MAINTAINERS | 1 +
> 4 files changed, 123 insertions(+), 19 deletions(-)
> create mode 100644 .github/workflows/build.yml
>
> diff --git a/.ci/linux-build.sh b/.ci/linux-build.sh
> index d079801d78..ee8d07f865 100755
> --- a/.ci/linux-build.sh
> +++ b/.ci/linux-build.sh
> @@ -12,7 +12,9 @@ on_error() {
> fi
> done
> }
> -trap on_error EXIT
> +# We capture the error logs as artifacts in Github Actions, no need to dump
> +# them via a EXIT handler.
> +[ -n "$GITHUB_WORKFLOW" ] || trap on_error EXIT
>
> install_libabigail() {
> version=$1
> @@ -28,16 +30,16 @@ install_libabigail() {
> rm ${version}.tar.gz
> }
>
> -if [ "$AARCH64" = "1" ]; then
> +if [ "$AARCH64" = "true" ]; then
> # convert the arch specifier
> OPTS="$OPTS --cross-file config/arm/arm64_armv8_linux_gcc"
> fi
>
> -if [ "$BUILD_DOCS" = "1" ]; then
> +if [ "$BUILD_DOCS" = "true" ]; then
> OPTS="$OPTS -Denable_docs=true"
> fi
>
> -if [ "$BUILD_32BIT" = "1" ]; then
> +if [ "$BUILD_32BIT" = "true" ]; then
> OPTS="$OPTS -Dc_args=-m32 -Dc_link_args=-m32"
> export PKG_CONFIG_LIBDIR="/usr/lib32/pkgconfig"
> fi
> @@ -48,16 +50,17 @@ else
> OPTS="$OPTS -Dexamples=all"
> fi
>
> +OPTS="$OPTS -Dmachine=default"
> OPTS="$OPTS --default-library=$DEF_LIB"
> OPTS="$OPTS --buildtype=debugoptimized"
> meson build --werror $OPTS
> ninja -C build
>
> -if [ "$AARCH64" != "1" ]; then
> +if [ "$AARCH64" != "true" ]; then
> devtools/test-null.sh
> fi
>
> -if [ "$ABI_CHECKS" = "1" ]; then
> +if [ "$ABI_CHECKS" = "true" ]; then
> LIBABIGAIL_VERSION=${LIBABIGAIL_VERSION:-libabigail-1.6}
>
> if [ "$(cat libabigail/VERSION 2>/dev/null)" != "$LIBABIGAIL_VERSION" ];
> then
> @@ -95,6 +98,6 @@ if [ "$ABI_CHECKS" = "1" ]; then
> devtools/check-abi.sh reference install ${ABI_CHECKS_WARN_ONLY:-}
> fi
>
> -if [ "$RUN_TESTS" = "1" ]; then
> +if [ "$RUN_TESTS" = "true" ]; then
> sudo meson test -C build --suite fast-tests -t 3
> fi
> diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
> new file mode 100644
> index 0000000000..bef6e52372
> --- /dev/null
> +++ b/.github/workflows/build.yml
> @@ -0,0 +1,100 @@
> +name: build
> +
> +on:
> + push:
> + schedule:
> + - cron: '0 0 * * 1'
> +
> +defaults:
> + run:
> + shell: bash --noprofile --norc -exo pipefail {0}
> +
> +jobs:
> + build:
> + name: ${{ join(matrix.config.*, '-') }}
> + runs-on: ${{ matrix.config.os }}
> + env:
> + AARCH64: ${{ matrix.config.cross == 'aarch64' }}
> + BUILD_32BIT: ${{ matrix.config.cross == 'i386' }}
> + BUILD_DOCS: ${{ contains(matrix.config.checks, 'doc') }}
> + CC: ccache ${{ matrix.config.compiler }}
> + DEF_LIB: ${{ matrix.config.library }}
> + RUN_TESTS: ${{ contains(matrix.config.checks, 'tests') }}
> +
> + strategy:
> + fail-fast: false
> + matrix:
> + config:
> + - os: ubuntu-18.04
> + compiler: gcc
> + library: static
> + - os: ubuntu-18.04
> + compiler: gcc
> + library: shared
> + checks: doc+tests
> + - os: ubuntu-18.04
> + compiler: clang
> + library: static
> + - os: ubuntu-18.04
> + compiler: clang
> + library: shared
> + checks: doc+tests
> + - os: ubuntu-18.04
> + compiler: gcc
> + library: static
> + cross: i386
> + - os: ubuntu-18.04
> + compiler: gcc
> + library: static
> + cross: aarch64
> + - os: ubuntu-18.04
> + compiler: gcc
> + library: shared
> + cross: aarch64
> +
> + steps:
> + - name: Checkout sources
> + uses: actions/checkout@v2
> + - name: Generate cache keys
> + id: get_ref_keys
> + run: |
> + echo -n '::set-output name=ccache::'
> + echo 'ccache-${{ matrix.config.os }}-${{ matrix.config.compiler
> }}-${{ matrix.config.cross }}-'$(date -u +%Y-w%W)
> + - name: Retrieve ccache cache
> + uses: actions/cache@v2
> + with:
> + path: ~/.ccache
> + key: ${{ steps.get_ref_keys.outputs.ccache }}-${{ github.ref }}
> + restore-keys: |
> + ${{ steps.get_ref_keys.outputs.ccache }}-refs/heads/main
> + - name: Install packages
> + run: sudo apt install -y ccache libnuma-dev python3-setuptools
> + python3-wheel python3-pip ninja-build libbsd-dev libpcap-dev
> + libibverbs-dev libcrypto++-dev libfdt-dev libjansson-dev
> + - name: Install i386 cross compiling packages
> + if: env.BUILD_32BIT == 'true'
> + run: sudo apt install -y gcc-multilib
> + - name: Install aarch64 cross compiling packages
> + if: env.AARCH64 == 'true'
> + run: sudo apt install -y gcc-aarch64-linux-gnu libc6-dev-arm64-cross
> + pkg-config-aarch64-linux-gnu
> + - name: Install doc generation packages
> + if: env.BUILD_DOCS == 'true'
> + run: sudo apt install -y doxygen graphviz python3-sphinx
> + python3-sphinx-rtd-theme
> + - name: Run setup
> + run: |
> + .ci/linux-setup.sh
> + # Workaround on $HOME permissions as EAL checks them for plugin
> loading
> + chmod o-w $HOME
> + - name: Build and test
> + run: .ci/linux-build.sh
> + - name: Upload logs on failure
> + if: failure()
> + uses: actions/upload-artifact@v2
> + with:
> + name: meson-logs-${{ join(matrix.config.*, '-') }}
> + path: |
> + build/meson-logs/testlog.txt
> + build/.ninja_log
> + build/meson-logs/meson-log.txt
> diff --git a/.travis.yml b/.travis.yml
> index 5e12db23b5..d655e286c3 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -34,10 +34,10 @@ jobs:
> - env: DEF_LIB="static"
> arch: amd64
> compiler: gcc
> - - env: DEF_LIB="shared" RUN_TESTS=1
> + - env: DEF_LIB="shared" RUN_TESTS=true
> arch: amd64
> compiler: gcc
> - - env: DEF_LIB="shared" BUILD_DOCS=1
> + - env: DEF_LIB="shared" BUILD_DOCS=true
> arch: amd64
> compiler: gcc
> addons:
> @@ -49,10 +49,10 @@ jobs:
> - env: DEF_LIB="static"
> arch: amd64
> compiler: clang
> - - env: DEF_LIB="shared" RUN_TESTS=1
> + - env: DEF_LIB="shared" RUN_TESTS=true
> arch: amd64
> compiler: clang
> - - env: DEF_LIB="shared" BUILD_DOCS=1
> + - env: DEF_LIB="shared" BUILD_DOCS=true
> arch: amd64
> compiler: clang
> addons:
> @@ -61,7 +61,7 @@ jobs:
> - *required_packages
> - *doc_packages
> # x86_64 cross-compiling 32-bits jobs
> - - env: DEF_LIB="static" BUILD_32BIT=1
> + - env: DEF_LIB="static" BUILD_32BIT=true
> arch: amd64
> compiler: gcc
> addons:
> @@ -69,14 +69,14 @@ jobs:
> packages:
> - *build_32b_packages
> # x86_64 cross-compiling aarch64 jobs
> - - env: DEF_LIB="static" AARCH64=1
> + - env: DEF_LIB="static" AARCH64=true
> arch: amd64
> compiler: gcc
> addons:
> apt:
> packages:
> - *aarch64_packages
> - - env: DEF_LIB="shared" AARCH64=1
> + - env: DEF_LIB="shared" AARCH64=true
> arch: amd64
> compiler: gcc
> addons:
> @@ -87,16 +87,16 @@ jobs:
> - env: DEF_LIB="static"
> arch: arm64
> compiler: gcc
> - - env: DEF_LIB="shared" RUN_TESTS=1
> + - env: DEF_LIB="shared" RUN_TESTS=true
> arch: arm64
> compiler: gcc
> - - env: DEF_LIB="shared" RUN_TESTS=1
> + - env: DEF_LIB="shared" RUN_TESTS=true
> dist: focal
> arch: arm64-graviton2
> virt: vm
> group: edge
> compiler: gcc
> - - env: DEF_LIB="shared" BUILD_DOCS=1
> + - env: DEF_LIB="shared" BUILD_DOCS=true
> arch: arm64
> compiler: gcc
> addons:
> @@ -108,10 +108,10 @@ jobs:
> - env: DEF_LIB="static"
> arch: arm64
> compiler: clang
> - - env: DEF_LIB="shared" RUN_TESTS=1
> + - env: DEF_LIB="shared" RUN_TESTS=true
> arch: arm64
> compiler: clang
> - - env: DEF_LIB="shared" RUN_TESTS=1
> + - env: DEF_LIB="shared" RUN_TESTS=true
> dist: focal
> arch: arm64-graviton2
> virt: vm
> diff --git a/MAINTAINERS b/MAINTAINERS
> index eafe9f8c46..f45c8c1b13 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -109,6 +109,7 @@ Public CI
> M: Aaron Conole <acon...@redhat.com>
> M: Michael Santana <maicolgabr...@hotmail.com>
> F: .travis.yml
> +F: .github/workflows/build.yml
> F: .ci/
>
> ABI Policy & Versioning
