On Wed, Mar 15, 2023 at 04:57:34PM +0100, Peter Eisentraut wrote:
> On 15.03.23 15:56, Justin Pryzby wrote:
> > I'm surprised if there's any question about the merits of making
> > documentation easily available for review. Several people have agreed;
> > one person mailed me privately specifically to ask how to show HTML docs
> > on cirrusci.
> >
> > Anyway, all this stuff is best addressed either before or after the CF.
> > I'll kick the patch forward. Thanks for looking.
>
> I suppose this depends on what you want to use this for. If your use is to
> prepare and lay out as much information as possible about a patch for a
> reviewer, some of your ideas make sense.
>
> I'm using this primarily to quickly test local work in progress. So I want
> a quick feedback cycle. I don't need it to show me which HTML docs changed,
> for example.
>
> So maybe there need to be different modes.
I'm opened to that - for example, mingw is currently opt-in. Maybe this
should be a separate task - it was implemented like that based on an
earlier suggestion (and then changed back again based on another
suggestion). The task could be triggered manually or by cfbot's
message.
But a primary goal for cirrus.yml was to allow developers to do the same
things as cfbot, and without everyone needing to reimplement it for
themselves.
You want quick feedback, like everyone else - but I doubt you disable
the documentation build when you don't need it, even though that would
shave off a whole minute. And I doubt that you'd comment it out even
the documentation was built twice.
Anyway - I think this patch is probably waiting on Andres' patch to
"convert CompilerWarnings to meson".
> > 7e09035f588 WIP: ci/meson: allow showing only failed tests ..
>
> I'm not sure I like this one. I sometimes look up the logs of non-failed
> tests to compare them with failed tests, to get context to could lead to
> failures. Maybe we can make this behavior adjustable. But I've not been
> bothered by the current behavior.
I suggest to try the patch; I doubt you'd prefer the existing behavior.
The patch is rebased now that meson is updated to avoid the windows
python warnings (thanks Andres).
--
Justin
>From 4695da5b426731a651d90fc41f41434207596848 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Wed, 25 May 2022 21:53:22 -0500
Subject: [PATCH 1/8] cirrus/windows: add compiler_warnings_script
The goal is to fail due to warnings only after running tests.
(At least historically, it's too slow to run a separate windows VM to
compile with -Werror.)
https://www.postgresql.org/message-id/20220212212310.f645c6vw3njkgxka%40alap3.anarazel.de
I'm not sure how to write this test in windows shell; it's also easy to
write something that doesn't work in posix sh, since windows shell is
interpretting && and ||...
See also:
8a1ce5e54f6d144e4f8e19af7c767b026ee0c956
https://cirrus-ci.com/task/6241060062494720
https://cirrus-ci.com/task/6496366607204352
ci-os-only: windows
---
.cirrus.yml | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index d3f88821a85..13213ffd304 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -589,28 +589,37 @@ task:
echo 127.0.0.2 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts
echo 127.0.0.3 pg-loadbalancetest >> c:\Windows\System32\Drivers\etc\hosts
type c:\Windows\System32\Drivers\etc\hosts
# Use /DEBUG:FASTLINK to avoid high memory usage during linking
configure_script: |
vcvarsall x64
meson setup --backend ninja --buildtype debug -Dc_link_args=/DEBUG:FASTLINK -Dcassert=true -Db_pch=true -Dextra_lib_dirs=c:\openssl\1.1\lib -Dextra_include_dirs=c:\openssl\1.1\include -DTAR=%TAR% -DPG_TEST_EXTRA="%PG_TEST_EXTRA%" build
build_script: |
vcvarsall x64
- ninja -C build
+ ninja -C build |tee build.txt
+ REM Since pipes lose the exit status of the preceding command, rerun the compilation
+ REM without the pipe, exiting now if it fails, to avoid trying to run checks
+ ninja -C build > nul
check_world_script: |
vcvarsall x64
meson test %MTEST_ARGS% --num-processes %TEST_JOBS%
+ # This should be last, so check_world is run even if there are warnings
+ always:
+ compiler_warnings_script:
+ # this avoids using metachars which would be interpretted by the windows shell
+ - sh -c 'if grep ": warning " build.txt; then exit 1; fi; exit 0'
+
on_failure:
<<: *on_failure_meson
crashlog_artifacts:
path: "crashlog-*.txt"
type: text/plain
task:
<< : *WINDOWS_ENVIRONMENT_BASE
name: Windows - Server 2019, MinGW64 - Meson
--
2.34.1
>From 62e04da9b12fc8a421957facee60d80c3c4f2ad8 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Fri, 24 Jun 2022 00:09:12 -0500
Subject: [PATCH 2/8] cirrus/freebsd: run with more CPUs+RAM and do not
repartition
There was some historic problem where tests under freebsd took 8+ minutes (and
before 4a288a37f took 15 minutes).
This reduces test time from 10min to 3min.
4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720
4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008
4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600
6 CPUs https://cirrus-ci.com/task/6678321684545536
8 CPUs https://cirrus-ci.com/task/6264854121021440
See also:
https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4h...@alap3.anarazel.de#f36c0b17e33e31e7925e7e5812998686
8 jobs 7min https://cirrus-ci.com/task/6186376667332608
//-os-only: freebsd
---
.cirrus.yml | 13 ++++---------
1 file changed, 4 insertions(+), 9 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 13213ffd304..21608ac2e0a 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -123,27 +123,25 @@ task:
<<: *on_failure_meson
cores_script: |
mkdir -m 770 /tmp/cores
find / -maxdepth 1 -type f -name 'core*' -exec mv '{}' /tmp/cores/ \;
src/tools/ci/cores_backtrace.sh linux /tmp/cores
task:
name: FreeBSD - 13 - Meson
env:
- # FreeBSD on GCP is slow when running with larger number of CPUS /
- # jobs. Using one more job than cpus seems to work best.
- CPUS: 2
- BUILD_JOBS: 3
- TEST_JOBS: 3
+ CPUS: 4
+ BUILD_JOBS: 4
+ TEST_JOBS: 6
CCACHE_DIR: /tmp/ccache_dir
CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST
CFLAGS: -Og -ggdb
depends_on: SanityCheck
only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*freebsd.*'
compute_engine_instance:
image_project: $IMAGE_PROJECT
image: family/pg-ci-freebsd-13
@@ -152,39 +150,36 @@ task:
memory: 4G
disk: 50
sysinfo_script: |
id
uname -a
ulimit -a -H && ulimit -a -S
export
ccache_cache:
folder: $CCACHE_DIR
- # Work around performance issues due to 32KB block size
- repartition_script: src/tools/ci/gcp_freebsd_repartition.sh
create_user_script: |
pw useradd postgres
chown -R postgres:postgres .
mkdir -p ${CCACHE_DIR}
chown -R postgres:postgres ${CCACHE_DIR}
setup_core_files_script: |
mkdir -m 770 /tmp/cores
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
#pkg install -y ...
# NB: Intentionally build without -Dllvm. The freebsd image size is already
- # large enough to make VM startup slow, and even without llvm freebsd
- # already takes longer than other platforms except for windows.
+ # large enough to make VM startup slow
configure_script: |
su postgres <<-EOF
meson setup \
--buildtype=debug \
-Dcassert=true -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS}'
upload_caches: ccache
--
2.34.1
>From 5c1691867c3098bb0ededb358d8ee329193b1bcb Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Fri, 9 Dec 2022 15:32:50 -0600
Subject: [PATCH 3/8] cirrus/freebsd: define
ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
See also: 54100f5c6052404f68de9ce7310ceb61f1c291f8
ci-os-only: freebsd
---
.cirrus.yml | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 21608ac2e0a..b9a70a05cfa 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -128,23 +128,23 @@ task:
task:
name: FreeBSD - 13 - Meson
env:
CPUS: 4
BUILD_JOBS: 4
TEST_JOBS: 6
CCACHE_DIR: /tmp/ccache_dir
- CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST
+ CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
CFLAGS: -Og -ggdb
depends_on: SanityCheck
only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*freebsd.*'
compute_engine_instance:
image_project: $IMAGE_PROJECT
image: family/pg-ci-freebsd-13
platform: freebsd
cpu: $CPUS
memory: 4G
--
2.34.1
>From b0cbff02c3a211f35df5be1ba63a5d55a3bfda8f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sat, 30 Jul 2022 19:25:51 -0500
Subject: [PATCH 4/8] cirrus/002_pg_upgrade: exercise --link and --clone
This increases code coverage (and maybe accelerates the test).
See also: b059a2409faf5833b3ba7792e247d6466c9e8090
linux,
macos,
//-os-only: freebsd
---
.cirrus.yml | 2 ++
1 file changed, 2 insertions(+)
diff --git a/.cirrus.yml b/.cirrus.yml
index b9a70a05cfa..4cc90484eaf 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -130,22 +130,23 @@ task:
task:
name: FreeBSD - 13 - Meson
env:
CPUS: 4
BUILD_JOBS: 4
TEST_JOBS: 6
CCACHE_DIR: /tmp/ccache_dir
CPPFLAGS: -DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST -DENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
CFLAGS: -Og -ggdb
+ PG_TEST_PG_UPGRADE_MODE: --link
depends_on: SanityCheck
only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*freebsd.*'
compute_engine_instance:
image_project: $IMAGE_PROJECT
image: family/pg-ci-freebsd-13
platform: freebsd
cpu: $CPUS
memory: 4G
disk: 50
@@ -424,22 +425,23 @@ task:
CIRRUS_WORKING_DIR: ${HOME}/pgsql/
CCACHE_DIR: ${HOME}/ccache
HOMEBREW_CACHE: ${HOME}/homebrew-cache
PERL5LIB: ${HOME}/perl5/lib/perl5
CC: ccache cc
CXX: ccache c++
CPPFLAGS: -DRANDOMIZE_ALLOCATED_MEMORY
CFLAGS: -Og -ggdb
CXXFLAGS: -Og -ggdb
+ PG_TEST_PG_UPGRADE_MODE: --clone
depends_on: SanityCheck
only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(macos|darwin|osx).*'
macos_instance:
image: ghcr.io/cirruslabs/macos-ventura-base:latest
sysinfo_script: |
id
uname -a
ulimit -a -H && ulimit -a -S
--
2.34.1
>From 0e079b20520fa001bb7dd10658acf061a3026733 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Fri, 25 Nov 2022 13:57:17 -0600
Subject: [PATCH 5/8] WIP: ci/meson: allow showing only failed tests ..
It's simpler and seems to make more sense to integrate this with
testwrap, rather than to run it after check-world, but only if it
failed, and finding a way to preserve the exit code.
https://www.postgresql.org/message-id/20221114235328.lxdj3puenfhir...@awork3.anarazel.de
> It is wasteful to upload thousdands of logfiles to show a single
> failure. That would make our cirrus tasks faster - compressing and
> uploading the logs takes over a minute.
>
> It's also a lot friendlier to show fewer than 8 pages of test folders to
> search through to find the one that failed.
macos
ci-os-only: linux-meson freebsd
---
.cirrus.yml | 13 +++++++------
src/tools/testwrap | 6 ++++++
2 files changed, 13 insertions(+), 6 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 4cc90484eaf..82ae18145b1 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -15,42 +15,43 @@ env:
CIRRUS_CLONE_DEPTH: 500
# Useful to be able to analyse what in a script takes long
CIRRUS_LOG_TIMESTAMP: true
CCACHE_MAXSIZE: "250M"
# target to test, for all but windows
CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS
CHECKFLAGS: -Otarget
PROVE_FLAGS: --timer
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
+ PG_FAILED_TESTDIR: ${CIRRUS_WORKING_DIR}/failed.build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
PG_TEST_EXTRA: kerberos ldap ssl load_balance
# What files to preserve in case tests fail
on_failure_ac: &on_failure_ac
log_artifacts:
paths:
- "**/*.log"
- "**/*.diffs"
- "**/regress_log_*"
type: text/plain
on_failure_meson: &on_failure_meson
testrun_artifacts:
paths:
- - "build*/testrun/**/*.log"
- - "build*/testrun/**/*.diffs"
- - "build*/testrun/**/regress_log_*"
+ - "failed.build*/**/*.log"
+ - "failed.build*/**/*.diffs"
+ - "failed.build*/**/regress_log_*"
type: text/plain
# In theory it'd be nice to upload the junit files meson generates, so that
# cirrus will nicely annotate the commit. Unfortunately the files don't
# contain identifiable file + line numbers right now, so the annotations
# don't end up useful. We could probably improve on that with a some custom
# conversion script, but ...
meson_log_artifacts:
path: "build*/meson-logs/*.txt"
type: text/plain
@@ -190,26 +191,26 @@ task:
ulimit -c unlimited
meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
EOF
# test runningcheck, freebsd chosen because it's currently fast enough
test_running_script: |
su postgres <<-EOF
set -e
ulimit -c unlimited
meson test $MTEST_ARGS --quiet --suite setup
export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH"
- mkdir -p build/testrun
+ mkdir -p build/testrun ${PG_FAILED_TESTDIR}
build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust
echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf
- build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l build/testrun/runningcheck.log start
+ build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l ${PG_FAILED_TESTDIR}/runningcheck.log start
meson test $MTEST_ARGS --num-processes ${TEST_JOBS} --setup running
build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop
EOF
on_failure:
# if the server continues running, it often causes cirrus-ci to fail
# during upload, as it doesn't expect artifacts to change size
stop_running_script: |
su postgres <<-EOF
build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop || true
EOF
@@ -394,23 +395,23 @@ task:
EOF
# so that we don't upload 64bit logs if 32bit fails
rm -rf build/
# There's currently no coverage of icu with LANG=C in the buildfarm. We
# can easily provide some here by running one of the sets of tests that
# way. Newer versions of python insist on changing the LC_CTYPE away
# from C, prevent that with PYTHONCOERCECLOCALE.
test_world_32_script: |
su postgres <<-EOF
ulimit -c unlimited
- PYTHONCOERCECLOCALE=0 LANG=C meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS}
+ PYTHONCOERCECLOCALE=0 LANG=C PG_FAILED_TESTDIR=`pwd`/failed.build-32 meson test $MTEST_ARGS -C build-32 --num-processes ${TEST_JOBS}
EOF
on_failure:
<<: *on_failure_meson
on_failure:
cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores
task:
name: macOS - Ventura - Meson
diff --git a/src/tools/testwrap b/src/tools/testwrap
index 7a64fe76a2d..445ac595afc 100755
--- a/src/tools/testwrap
+++ b/src/tools/testwrap
@@ -36,12 +36,18 @@ env_dict = {**os.environ,
'TESTDATADIR': os.path.join(testdir, 'data'),
'TESTLOGDIR': os.path.join(testdir, 'log')}
sp = subprocess.run(args.test_command, env=env_dict)
if sp.returncode == 0:
print('# test succeeded')
open(os.path.join(testdir, 'test.success'), 'x')
else:
print('# test failed')
open(os.path.join(testdir, 'test.fail'), 'x')
+ faileddir = os.getenv('PG_FAILED_TESTDIR')
+ if faileddir:
+ parentdir = os.path.dirname(testdir)
+ newdest = os.path.join(faileddir, os.path.basename(parentdir), os.path.basename(testdir))
+ shutil.copytree(testdir, newdest)
+
sys.exit(sp.returncode)
--
2.34.1
>From 651fce87b7bda15029c5b55c4fca2a1e63dfaf29 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Mon, 17 Jan 2022 00:54:28 -0600
Subject: [PATCH 6/8] cirrus: show coverage report of new code for every patch
Coverage is shown only for changed files. This is useful to see
coverage of newly-added code, but won't show added/lost coverage in
files which this patch doesn't modify.
Some alternatives:
- could build with "--coverage -fprofile-filter-files=", but that means
that ccache will never work (both because it doesn't support that option
and also because the arguments will be different for every patch).
- could use ninja coverage-html, but that can't filter only changed
files, and would take a long time and a lot of space to upload a lot
of useless files.
https://www.postgresql.org/message-id/202202111821.w3gqblvfp4pr%40alvherre.pgsql
https://www.postgresql.org/message-id/flat/20220409021853.gp24...@telsasoft.com
ci-os-only: freebsd
---
.cirrus.yml | 20 ++++++++++++-
src/tools/ci/code-coverage-report | 48 +++++++++++++++++++++++++++++++
2 files changed, 67 insertions(+), 1 deletion(-)
create mode 100755 src/tools/ci/code-coverage-report
diff --git a/.cirrus.yml b/.cirrus.yml
index 82ae18145b1..0f48bd2ab9d 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -20,22 +20,30 @@ env:
# target to test, for all but windows
CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS
CHECKFLAGS: -Otarget
PROVE_FLAGS: --timer
MTEST_ARGS: --print-errorlogs --no-rebuild -C build
PG_FAILED_TESTDIR: ${CIRRUS_WORKING_DIR}/failed.build
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
PG_TEST_EXTRA: kerberos ldap ssl load_balance
+ # The commit that this branch is rebased on. There's no easy way to find this.
+ # This does the right thing for cfbot, which always squishes all patches into a single commit.
+ # And does the right thing for any 1-patch commits.
+ # Patch series manually submitted to cirrus would benefit from setting this to the
+ # number of patches in the series (or directly to the commit the series was rebased on).
+ #BASE_COMMIT: HEAD~1
+ # For demo purposes:
+ BASE_COMMIT: HEAD~11
# What files to preserve in case tests fail
on_failure_ac: &on_failure_ac
log_artifacts:
paths:
- "**/*.log"
- "**/*.diffs"
- "**/regress_log_*"
type: text/plain
on_failure_meson: &on_failure_meson
@@ -149,57 +157,67 @@ task:
image: family/pg-ci-freebsd-13
platform: freebsd
cpu: $CPUS
memory: 4G
disk: 50
sysinfo_script: |
id
uname -a
ulimit -a -H && ulimit -a -S
export
+ git diff --name-only "$BASE_COMMIT"
ccache_cache:
folder: $CCACHE_DIR
create_user_script: |
pw useradd postgres
chown -R postgres:postgres .
mkdir -p ${CCACHE_DIR}
chown -R postgres:postgres ${CCACHE_DIR}
setup_core_files_script: |
mkdir -m 770 /tmp/cores
chown root:postgres /tmp/cores
sysctl kern.corefile='/tmp/cores/%N.%P.core'
setup_additional_packages_script: |
- #pkg install -y ...
+ pkg install -y lcov
# NB: Intentionally build without -Dllvm. The freebsd image size is already
# large enough to make VM startup slow
configure_script: |
su postgres <<-EOF
meson setup \
--buildtype=debug \
+ -Db_coverage=true \
-Dcassert=true -Duuid=bsd -Dtcl_version=tcl86 -Ddtrace=auto \
-DPG_TEST_EXTRA="$PG_TEST_EXTRA" \
-Dextra_lib_dirs=/usr/local/lib -Dextra_include_dirs=/usr/local/include/ \
build
EOF
build_script: su postgres -c 'ninja -C build -j${BUILD_JOBS}'
upload_caches: ccache
test_world_script: |
su postgres <<-EOF
+ set -e
ulimit -c unlimited
+ # Write initial coverage files before running tests:
+ time ./src/tools/ci/code-coverage-report "$BASE_COMMIT" ./build ./coverage "--initial"
meson test $MTEST_ARGS --num-processes ${TEST_JOBS}
+ # Create coverage report for files changed since the base commit.
+ time ./src/tools/ci/code-coverage-report "$BASE_COMMIT" ./build ./coverage
EOF
+ coverage_artifacts:
+ path: 'coverage/**'
+
# test runningcheck, freebsd chosen because it's currently fast enough
test_running_script: |
su postgres <<-EOF
set -e
ulimit -c unlimited
meson test $MTEST_ARGS --quiet --suite setup
export LD_LIBRARY_PATH="$(pwd)/build/tmp_install/usr/local/pgsql/lib/:$LD_LIBRARY_PATH"
mkdir -p build/testrun ${PG_FAILED_TESTDIR}
build/tmp_install/usr/local/pgsql/bin/initdb -N build/runningcheck --no-instructions -A trust
echo "include '$(pwd)/src/tools/ci/pg_ci_base.conf'" >> build/runningcheck/postgresql.conf
build/tmp_install/usr/local/pgsql/bin/pg_ctl -c -o '-c fsync=off' -D build/runningcheck -l ${PG_FAILED_TESTDIR}/runningcheck.log start
diff --git a/src/tools/ci/code-coverage-report b/src/tools/ci/code-coverage-report
new file mode 100755
index 00000000000..db448e802ba
--- /dev/null
+++ b/src/tools/ci/code-coverage-report
@@ -0,0 +1,48 @@
+#! /bin/sh
+# Called during CI to generate a code coverage report of changed files.
+set -e
+
+base_branch=$1
+build_dir=$2
+outdir=$3
+args=$4
+
+changed=`git diff --name-only "$base_branch" '*.c'`
+[ -z "$changed" ] && exit 0 # Nothing changed
+
+[ -d "$outdir" ] ||
+ mkdir "$outdir"
+
+# This could be used to map from object file back to source file:
+# readelf --debug-dump=info src/backend/postgres_lib.a.p/utils_adt_array_userfuncs.c.o |awk '/DW_AT_name/{print $NF;exit}'
+# gcov ./src/port/libpgport_shlib.a.p/inet_net_ntop.c.gcno --stdout
+
+gcov=$outdir/coverage.gcov
+lcov --quiet --capture --directory "$build_dir" $args >"$gcov.new"
+
+# Filter to include only changed files
+echo "$changed" |sed 's,^,*/,' |
+ xargs -rt lcov --extract "$gcov.new" >"$gcov.filtered"
+rm "$gcov.new"
+
+echo "$args" |grep initial >/dev/null && {
+ mv "$gcov.filtered" "$gcov.init"
+ exit 0
+}
+
+# Exit successfully if no relevant files were changed
+[ -s "$gcov.filtered" ] || {
+ rm "$gcov.init"
+ exit 0
+}
+
+# Otherwise combine with the init file:
+lcov -a "$gcov.init" -a "$gcov.filtered" >"$gcov"
+
+genhtml "$gcov" --show-details --legend --quiet --num-spaces=4 --output-directory "$outdir" \
+ --title="Coverage report of files changed since: $base_branch"
+cp "$outdir"/index.html "$outdir"/00-index.html
+
+gzip "$gcov" "$gcov.init" "$gcov.filtered"
+ls -l "$outdir"
+du -sh "$outdir"
--
2.34.1
>From a18875c0da01c2a9cd9a7e0a9b9edbe88883129c Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sat, 26 Feb 2022 19:39:10 -0600
Subject: [PATCH 7/8] cirrus: upload changed html docs as artifacts
This could be done on the client side (cfbot). One advantage of doing
it here is that fewer docs are uploaded - many patches won't upload docs
at all.
https://www.postgresql.org/message-id/flat/20220409021853.gp24...@telsasoft.com
https://www.postgresql.org/message-id/CAB8KJ=i4qmeuopq+pcsmbzgd4o-xv0fcnc+q1x7hn9hsdvk...@mail.gmail.com
https://cirrus-ci.com/task/5396696388599808
ci-os-only: html
---
.cirrus.yml | 16 +++++++++++++++-
src/tools/ci/copy-changed-docs | 29 +++++++++++++++++++++++++++++
2 files changed, 44 insertions(+), 1 deletion(-)
create mode 100755 src/tools/ci/copy-changed-docs
diff --git a/.cirrus.yml b/.cirrus.yml
index 0f48bd2ab9d..68b7229d117 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -725,22 +725,23 @@ task:
image: $CONTAINER_REPO/linux_debian_bullseye_ci:latest
cpu: $CPUS
sysinfo_script: |
id
uname -a
cat /proc/cmdline
ulimit -a -H && ulimit -a -S
gcc -v
clang -v
export
+ git diff --name-only "$BASE_COMMIT"
ccache_cache:
folder: $CCACHE_DIR
setup_additional_packages_script: |
#apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y install ...
###
# Test that code can be built with gcc/clang without warnings
###
@@ -799,34 +800,47 @@ task:
always:
mingw_cross_warning_script: |
time ./configure \
--host=x86_64-w64-mingw32 \
--enable-cassert \
CC="ccache x86_64-w64-mingw32-gcc" \
CXX="ccache x86_64-w64-mingw32-g++"
make -s -j${BUILD_JOBS} clean
time make -s -j${BUILD_JOBS} world-bin
###
- # Verify docs can be built
+ # Verify docs can be built, and upload changed docs as artifacts
###
# XXX: Only do this if there have been changes in doc/ since last build
always:
docs_build_script: |
time ./configure \
--cache gcc.cache \
CC="ccache gcc" \
CXX="ccache g++" \
CLANG="ccache clang"
make -s -j${BUILD_JOBS} clean
time make -s -j${BUILD_JOBS} -C doc
+ cp -r doc new-docs
+
+ # Re-build HTML docs from the base commit.
+ git checkout "$BASE_COMMIT" -- doc
+ make -s -C doc clean
+ time make -s -C doc html
+ cp -r doc old-docs
+
+ copy_changed_docs_script:
+ - src/tools/ci/copy-changed-docs "old-docs" "new-docs" "html_docs"
+
+ html_docs_artifacts:
+ paths: ['html_docs/*.html', 'html_docs/*.png', 'html_docs/*.css']
###
# Verify headerscheck / cpluspluscheck succeed
#
# - Don't use ccache, the files are uncacheable, polluting ccache's
# cache
# - Use -fmax-errors, as particularly cpluspluscheck can be very verbose
# - XXX have to disable ICU to avoid errors:
# https://postgr.es/m/20220323002024.f2g6tivduzrktgfa%40alap3.anarazel.de
###
always:
diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs
new file mode 100755
index 00000000000..1c921a8df6f
--- /dev/null
+++ b/src/tools/ci/copy-changed-docs
@@ -0,0 +1,29 @@
+#! /bin/sh
+# Copy HTML which differ between $old and $new into $outdir
+set -e
+
+old=$1
+new=$2
+outdir=$3
+
+# The index is large and changes often
+skippages="bookindex.html"
+
+mkdir "$outdir"
+cp "$new"/src/sgml/html/*.css "$new"/src/sgml/html/*.svg "$outdir"
+
+changed=`git diff --no-index --name-only "$old"/src/sgml/html "$new"/src/sgml/html` ||
+ [ $? -eq 1 ]
+
+for f in $changed
+do
+ # Avoid removed files
+ [ -f "$f" ] || continue
+
+ echo "$f" |grep -Ew "$skippages" >/dev/null &&
+ continue
+
+ cp -v "$f" "$outdir"
+done
+
+exit 0
--
2.34.1
>From 5287027a1bceb719075a681fc130c246dc2a5d38 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Mon, 28 Feb 2022 23:18:19 -0600
Subject: [PATCH 8/8] +html index file
This allows linking to the artifacts from the last successful build.
//freebsd
ci-os-only: warnings
---
.cirrus.yml | 4 ++--
src/tools/ci/copy-changed-docs | 39 ++++++++++++++++++++++++++++++++--
2 files changed, 39 insertions(+), 4 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 68b7229d117..e6f435b6981 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -27,23 +27,23 @@ env:
PGCTLTIMEOUT: 120 # avoids spurious failures during parallel tests
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
PG_TEST_EXTRA: kerberos ldap ssl load_balance
# The commit that this branch is rebased on. There's no easy way to find this.
# This does the right thing for cfbot, which always squishes all patches into a single commit.
# And does the right thing for any 1-patch commits.
# Patch series manually submitted to cirrus would benefit from setting this to the
# number of patches in the series (or directly to the commit the series was rebased on).
#BASE_COMMIT: HEAD~1
# For demo purposes:
- BASE_COMMIT: HEAD~11
+ BASE_COMMIT: HEAD~22
# What files to preserve in case tests fail
on_failure_ac: &on_failure_ac
log_artifacts:
paths:
- "**/*.log"
- "**/*.diffs"
- "**/regress_log_*"
type: text/plain
on_failure_meson: &on_failure_meson
@@ -699,23 +699,23 @@ task:
type: text/plain
task:
name: CompilerWarnings
# To limit unnecessary work only run this once the SanityCheck
# succeeds. This is particularly important for this task as we intentionally
# use always: to continue after failures. Task that did not run count as a
# success, so we need to recheck SanityChecks's condition here ...
depends_on: SanityCheck
- only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*'
+ # only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*'
env:
CPUS: 4
BUILD_JOBS: 4
# Use larger ccache cache, as this task compiles with multiple compilers /
# flag combinations
CCACHE_MAXSIZE: "1G"
CCACHE_DIR: "/tmp/ccache_dir"
LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES
diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs
index 1c921a8df6f..0efad386cca 100755
--- a/src/tools/ci/copy-changed-docs
+++ b/src/tools/ci/copy-changed-docs
@@ -1,29 +1,64 @@
#! /bin/sh
# Copy HTML which differ between $old and $new into $outdir
set -e
old=$1
new=$2
outdir=$3
+branch=$CIRRUS_BRANCH
# The index is large and changes often
skippages="bookindex.html"
mkdir "$outdir"
cp "$new"/src/sgml/html/*.css "$new"/src/sgml/html/*.svg "$outdir"
+# The index is useful to allow a static link (not specific to a cirrus run) to the artifacts for the most-recent, successful CI run for a branch
+branchurl=https://api.cirrus-ci.com/v1/artifact/github/$CIRRUS_REPO_FULL_NAME/$CIRRUS_TASK_NAME/html_docs/html_docs/00-index.html?branch=$branch
+
+index="$outdir/00-index.html"
+cat >"$index" <<EOF
+<html>
+<head><title>Index of docs changed since: $BASE_COMMIT</title></head>
+<body>
+<!-- A link to documentation for the most recent successful build: $branchurl -->
+<h1>Index of docs changed since: $BASE_COMMIT</h1>
+<ul>
+EOF
+
changed=`git diff --no-index --name-only "$old"/src/sgml/html "$new"/src/sgml/html` ||
[ $? -eq 1 ]
for f in $changed
do
# Avoid removed files
[ -f "$f" ] || continue
echo "$f" |grep -Ew "$skippages" >/dev/null &&
continue
- cp -v "$f" "$outdir"
-done
+ cp -v "$f" "$outdir" >&2
+ fn=${f##*/}
+ # ?branch=... is needed when accessing the artifacts for the static link for the branch
+ # It's not needed and ignored if accessing artifacts for *this* CI run
+ echo "<li><a href='$fn?branch=$branch'>$fn</a>"
+done >>"$index"
+
+github=https://github.com/$CIRRUS_REPO_FULL_NAME/commit
+cirrus=https://cirrus-ci.com/build
+
+cat >>"$index" <<EOF
+</ul>
+<hr>
+<code>
+<br>This file was written on: `date --rfc-822 --utc`
+<br>CIRRUS_CHANGE_TITLE: $CIRRUS_CHANGE_TITLE
+<br>CIRRUS_CHANGE_IN_REPO: <a href="$github/$CIRRUS_CHANGE_IN_REPO">$CIRRUS_CHANGE_IN_REPO</a>
+<br>CIRRUS_BUILD_ID: <a href="$cirrus/$CIRRUS_BUILD_ID">$CIRRUS_BUILD_ID</a>
+<br>CIRRUS_LAST_GREEN_CHANGE: <a href="$github/$CIRRUS_LAST_GREEN_CHANGE">$CIRRUS_LAST_GREEN_CHANGE</a>
+<br>CIRRUS_LAST_GREEN_BUILD_ID: <a href="$cirrus/$CIRRUS_LAST_GREEN_BUILD_ID">$CIRRUS_LAST_GREEN_BUILD_ID</a>
+</code>
+</body></html>
+EOF
exit 0
--
2.34.1
