Resending with a problematic address removed...
On Sat, May 28, 2022 at 10:37:41AM -0500, Justin Pryzby wrote:
> I'm anticipating the need to further re-arrange the patch set - it's not clear
> which patches should go first. Maybe some patches should be dropped in favour
> of the meson project. I guess some patches will have to be re-implemented
> with
> meson (msvc warnings).
> Maybe the improvements to vcregress should go into v15 ? CI should run all
> the
> tests (which also serves to document *how* to run all the tests, even if there
> isn't a literal check-world target).
On Thu, Jun 23, 2022 at 02:31:25PM -0500, Justin Pryzby wrote:
> Should any of the test changes go into v15 ?
On Thu, Jul 07, 2022 at 07:22:32PM -0500, Justin Pryzby wrote:
> Also, cirrus/freebsd task can run 3x faster with more CPUs.
Checking if there's interest in any/none of these patches ?
I have added several more.
Do you have an idea when the meson branch might be merged ?
Will vcregress remain for a while, or will it go away for v16 ?
--
Justin
>From 99ee0bef5054539aad0e23a24dd9c9cc9ccee788 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Wed, 25 May 2022 21:53:22 -0500
Subject: [PATCH 01/25] cirrus/windows: add compiler_warnings_script
I'm not sure how to write this test in windows shell; it's also not easy to
write it in posix sh, since windows shell is somehow interpretting && and ||...
https://www.postgresql.org/message-id/20220212212310.f645c6vw3njkgxka%40alap3.anarazel.de
See also: 8a1ce5e54f6d144e4f8e19af7c767b026ee0c956
ci-os-only: windows
https://cirrus-ci.com/task/6183879907213312
https://cirrus-ci.com/task/4876271443247104
---
.cirrus.yml | 8 +++++++-
src/tools/ci/windows-compiler-warnings | 16 ++++++++++++++++
2 files changed, 23 insertions(+), 1 deletion(-)
create mode 100755 src/tools/ci/windows-compiler-warnings
diff --git a/.cirrus.yml b/.cirrus.yml
index 81eb8a9996d..da16344341b 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -370,7 +370,8 @@ task:
# ForceNoAlign prevents msbuild from introducing line-breaks for long lines
# disable file tracker, we're never going to rebuild, and it slows down the
# build
- MSBFLAGS: -m -verbosity:minimal "-consoleLoggerParameters:Summary;ForceNoAlign" /p:TrackFileAccess=false -nologo
+ # -fileLoggerParameters1: write warnings to msbuild.warn.log.
+ MSBFLAGS: -m -verbosity:minimal "-consoleLoggerParameters:Summary;ForceNoAlign" /p:TrackFileAccess=false -nologo -fileLoggerParameters1:warningsonly;logfile=msbuild.warn.log
# If tests hang forever, cirrus eventually times out. In that case log
# output etc is not uploaded, making the problem hard to debug. Of course
@@ -450,6 +451,11 @@ task:
cd src/tools/msvc
%T_C% perl vcregress.pl ecpgcheck
+ # These should be last, so all the important checks are always run
+ always:
+ compiler_warnings_script:
+ - sh src\tools\ci\windows-compiler-warnings msbuild.warn.log
+
on_failure:
<<: *on_failure
crashlog_artifacts:
diff --git a/src/tools/ci/windows-compiler-warnings b/src/tools/ci/windows-compiler-warnings
new file mode 100755
index 00000000000..d6f9a1fc569
--- /dev/null
+++ b/src/tools/ci/windows-compiler-warnings
@@ -0,0 +1,16 @@
+#! /bin/sh
+# Success if the given file doesn't exist or is empty, else fail
+# This is a separate file only to avoid dealing with windows shell quoting and escaping.
+set -e
+
+fn=$1
+
+if [ -s "$fn" ]
+then
+ # Display the file's content, then exit indicating failure
+ cat "$fn"
+ exit 1
+else
+ # Success
+ exit 0
+fi
--
2.17.1
>From 278c842aa7021c2c551102f0023adff3a7bbd495 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sun, 9 Jan 2022 18:25:02 -0600
Subject: [PATCH 02/25] cirrus/vcregress: test modules/contrib with
NO_INSTALLCHECK=1
--temp-config must be specified with an "=" because otherwise vcregress runs
pg_regress --temp-config test1 test2 [...],
..which means test1 gets eaten as the argument to --temp-config
https://www.postgresql.org/message-id/20220109191649.GL14051%40telsasoft.com
https://www.postgresql.org/message-id/CA%2BhUKGLneD%2Bq%2BE7upHGwn41KGvbxhsKbJ%2BM-y9nvv7_Xjv8Qog%40mail.gmail.com
---
.cirrus.yml | 4 +-
contrib/basic_archive/Makefile | 2 +-
contrib/pg_stat_statements/Makefile | 2 +-
contrib/test_decoding/Makefile | 2 +-
src/test/modules/snapshot_too_old/Makefile | 2 +-
src/test/modules/worker_spi/Makefile | 2 +-
src/tools/msvc/vcregress.pl | 46 +++++++++++++++++++---
7 files changed, 48 insertions(+), 12 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index da16344341b..f2861176be2 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -428,9 +428,9 @@ task:
test_isolation_script: |
%T_C% perl src/tools/msvc/vcregress.pl isolationcheck
test_modules_script: |
- %T_C% perl src/tools/msvc/vcregress.pl modulescheck
+ %T_C% perl src/tools/msvc/vcregress.pl modulescheck install
test_contrib_script: |
- %T_C% perl src/tools/msvc/vcregress.pl contribcheck
+ %T_C% perl src/tools/msvc/vcregress.pl contribcheck install
stop_script: |
tmp_install\bin\pg_ctl.exe stop -D tmp_check/db -l tmp_check/postmaster.log
test_ssl_script: |
diff --git a/contrib/basic_archive/Makefile b/contrib/basic_archive/Makefile
index 14d036e1c42..246358973fe 100644
--- a/contrib/basic_archive/Makefile
+++ b/contrib/basic_archive/Makefile
@@ -4,7 +4,7 @@ MODULES = basic_archive
PGFILEDESC = "basic_archive - basic archive module"
REGRESS = basic_archive
-REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/basic_archive/basic_archive.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/contrib/basic_archive/basic_archive.conf
NO_INSTALLCHECK = 1
diff --git a/contrib/pg_stat_statements/Makefile b/contrib/pg_stat_statements/Makefile
index edc40c8bbfb..8684ea0be9c 100644
--- a/contrib/pg_stat_statements/Makefile
+++ b/contrib/pg_stat_statements/Makefile
@@ -16,7 +16,7 @@ PGFILEDESC = "pg_stat_statements - execution statistics of SQL statements"
LDFLAGS_SL += $(filter -lm, $(LIBS))
-REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/pg_stat_statements/pg_stat_statements.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/contrib/pg_stat_statements/pg_stat_statements.conf
REGRESS = pg_stat_statements oldextversions
# Disabled because these tests require "shared_preload_libraries=pg_stat_statements",
# which typical installcheck users do not have (e.g. buildfarm clients).
diff --git a/contrib/test_decoding/Makefile b/contrib/test_decoding/Makefile
index c7ce6037064..a6289f606d3 100644
--- a/contrib/test_decoding/Makefile
+++ b/contrib/test_decoding/Makefile
@@ -10,7 +10,7 @@ ISOLATION = mxact delayed_startup ondisk_startup concurrent_ddl_dml \
oldest_xmin snapshot_transfer subxact_without_top concurrent_stream \
twophase_snapshot slot_creation_error catalog_change_snapshot
-REGRESS_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/contrib/test_decoding/logical.conf
ISOLATION_OPTS = --temp-config $(top_srcdir)/contrib/test_decoding/logical.conf
# Disabled because these tests require "wal_level=logical", which
diff --git a/src/test/modules/snapshot_too_old/Makefile b/src/test/modules/snapshot_too_old/Makefile
index dfb4537f63c..752a0039fdc 100644
--- a/src/test/modules/snapshot_too_old/Makefile
+++ b/src/test/modules/snapshot_too_old/Makefile
@@ -5,7 +5,7 @@
EXTRA_CLEAN = $(pg_regress_clean_files)
ISOLATION = sto_using_cursor sto_using_select sto_using_hash_index
-ISOLATION_OPTS = --temp-config $(top_srcdir)/src/test/modules/snapshot_too_old/sto.conf
+ISOLATION_OPTS = --temp-config=$(top_srcdir)/src/test/modules/snapshot_too_old/sto.conf
# Disabled because these tests require "old_snapshot_threshold" >= 0, which
# typical installcheck users do not have (e.g. buildfarm clients).
diff --git a/src/test/modules/worker_spi/Makefile b/src/test/modules/worker_spi/Makefile
index cbf9b2e37fd..d9f7d9bab6d 100644
--- a/src/test/modules/worker_spi/Makefile
+++ b/src/test/modules/worker_spi/Makefile
@@ -9,7 +9,7 @@ PGFILEDESC = "worker_spi - background worker example"
REGRESS = worker_spi
# enable our module in shared_preload_libraries for dynamic bgworkers
-REGRESS_OPTS = --temp-config $(top_srcdir)/src/test/modules/worker_spi/dynamic.conf
+REGRESS_OPTS = --temp-config=$(top_srcdir)/src/test/modules/worker_spi/dynamic.conf
# Disable installcheck to ensure we cover dynamic bgworkers.
NO_INSTALLCHECK = 1
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index c3729f6be5e..2d6ccd45419 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -426,6 +426,7 @@ sub plcheck
sub subdircheck
{
my $module = shift;
+ my $installcheck = shift || 1;
if ( !-d "$module/sql"
|| !-d "$module/expected"
@@ -435,7 +436,7 @@ sub subdircheck
}
chdir $module;
- my @tests = fetchTests();
+ my @tests = fetchTests($installcheck);
# Leave if no tests are listed in the module.
if (scalar @tests == 0)
@@ -445,12 +446,13 @@ sub subdircheck
}
my @opts = fetchRegressOpts();
+ push @opts, "--temp-instance=tmp_check" if $installcheck == -1;
print "============================================================\n";
print "Checking $module\n";
my @args = (
"$topdir/$Config/pg_regress/pg_regress",
- "--bindir=${topdir}/${Config}/psql",
+ "--bindir=$tmp_installdir/bin",
"--dbname=contrib_regression", @opts, @tests);
print join(' ', @args), "\n";
system(@args);
@@ -460,6 +462,8 @@ sub subdircheck
sub contribcheck
{
+ my $mode = shift || '';
+
chdir "../../../contrib";
my $mstat = 0;
foreach my $module (glob("*"))
@@ -477,12 +481,25 @@ sub contribcheck
my $status = $? >> 8;
$mstat ||= $status;
}
+
+ # As above, but creates new DB instance for each module. For CI.
+ if ($mode eq "install")
+ {
+ foreach my $module (glob("*"))
+ {
+ subdircheck("$module", -1);
+ $mstat ||= $? >> 8;
+ }
+ }
+
exit $mstat if $mstat;
return;
}
sub modulescheck
{
+ my $mode = shift || '';
+
chdir "../../../src/test/modules";
my $mstat = 0;
foreach my $module (glob("*"))
@@ -491,6 +508,17 @@ sub modulescheck
my $status = $? >> 8;
$mstat ||= $status;
}
+
+ # As above, but creates new DB instance for each module. For CI.
+ if ($mode eq "install")
+ {
+ foreach my $module (glob("*"))
+ {
+ subdircheck("$module", -1);
+ $mstat ||= $? >> 8;
+ }
+ }
+
exit $mstat if $mstat;
return;
}
@@ -558,6 +586,7 @@ sub fetchRegressOpts
# option starting with "--".
@opts = grep { !/\$\(/ && /^--/ }
map { (my $x = $_) =~ s/\Q$(top_builddir)\E/\"$topdir\"/; $x; }
+ map { (my $x = $_) =~ s/\Q$(top_srcdir)\E/\"$topdir\"/; $x; }
split(/\s+/, $1);
}
if ($m =~ /^\s*ENCODING\s*=\s*(\S+)/m)
@@ -583,14 +612,19 @@ sub fetchTests
my $m = <$handle>;
close($handle);
my $t = "";
+ my $installcheck = shift || 1;
$m =~ s{\\\r?\n}{}g;
- # A module specifying NO_INSTALLCHECK does not support installcheck,
- # so bypass its run by returning an empty set of tests.
if ($m =~ /^\s*NO_INSTALLCHECK\s*=\s*\S+/m)
{
- return ();
+ # Skip modules marked installcheck unless running installcheck tests.
+ return () if $installcheck == 1;
+ }
+ else
+ {
+ # Skip modules not marked installcheck if running installcheck tests.
+ return () if $installcheck == -1;
}
if ($m =~ /^REGRESS\s*=\s*(.*)$/gm)
@@ -656,6 +690,8 @@ sub usage
"\nOptions for <arg>: (used by check and installcheck)\n",
" serial serial mode\n",
" parallel parallel mode\n",
+ "\nOptions for <arg>: (used by contribcheck and modulescheck)\n",
+ " install also run tests which require a new instance\n",
"\nOption for <arg>: for taptest\n",
" TEST_DIR (required) directory where tests reside\n";
exit(1);
--
2.17.1
>From 1064a0794e85e06b3a0eca4ed3765f078795cb36 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sun, 3 Apr 2022 00:10:20 -0500
Subject: [PATCH 03/25] cirrus/ccache: disable compression and show stats
ccache since 4.0 enables zstd compression by default.
With default compression enabled (https://cirrus-ci.com/task/6692342840164352):
linux/debian/bullseye has 4.2; 99MB cirrus cache; cache_size_kibibyte 109616
macos has 4.5.1: 47MB cirrus cache; cache_size_kibibyte 52500
freebsd has 3.7.12: 42MB cirrus cache; cache_size_kibibyte 134064
windows has 4.6.1; 180MB cirrus cache; cache_size_kibibyte 51179
todo: compiler warnings
With compression disabled (https://cirrus-ci.com/task/4614182514458624):
linux: 91MB cirrus cache; cache_size_kibibyte 316136
macos: 41MB cirrus cache; cache_size_kibibyte 118068
windows: 42MB cirrus cache; cache_size_kibibyte 134064
freebsd is the same
The stats should either be shown or logged (or maybe run with CCACHE_NOSTATS,
to avoid re-uploading cache tarball in a 100% cached build, due only to
updating ./**/stats).
Note that ccache 4.4 supports CCACHE_STATSLOG, which seems ideal.
freebsd, linux
ci-os-only: macos
---
.cirrus.yml | 33 ++++++++++++++++++++++++++++-----
1 file changed, 28 insertions(+), 5 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index f2861176be2..c0616e8560d 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -16,7 +16,9 @@ env:
# Useful to be able to analyse what in a script takes long
CIRRUS_LOG_TIMESTAMP: true
- CCACHE_MAXSIZE: "250M"
+ CCACHE_MAXSIZE: "500M"
+ CCACHE_NOCOMPRESS: 1
+ #CCACHE_STATSLOG: ccache-stats.log
# target to test, for all but windows
CHECK: check-world PROVE_FLAGS=$PROVE_FLAGS
@@ -110,7 +112,10 @@ task:
CXX="ccache c++" \
CFLAGS="-Og -ggdb"
EOF
- build_script: su postgres -c "gmake -s -j${BUILD_JOBS} world-bin"
+ build_script: |
+ su postgres -c "ccache --zero-stats"
+ su postgres -c "gmake -s -j${BUILD_JOBS} world-bin"
+ su postgres -c "ccache --show-stats"
upload_caches: ccache
# The use of script avoids make -Otarget complaints about fcntl() on
@@ -206,7 +211,11 @@ task:
CFLAGS="-Og -ggdb" \
CXXFLAGS="-Og -ggdb"
EOF
- build_script: su postgres -c "make -s -j${BUILD_JOBS} world-bin"
+
+ build_script: |
+ su postgres -c "ccache --zero-stats"
+ su postgres -c "make -s -j${BUILD_JOBS} world-bin"
+ su postgres -c "ccache --show-stats"
upload_caches: ccache
test_world_script: |
@@ -324,7 +333,11 @@ task:
\
LLVM_CONFIG=${brewpath}/llvm/bin/llvm-config \
PYTHON=python3
- build_script: gmake -s -j${BUILD_JOBS} world-bin
+
+ build_script: |
+ ccache --zero-stats
+ gmake -s -j${BUILD_JOBS} world-bin
+ ccache --show-stats -v
upload_caches: ccache
test_world_script: |
@@ -476,7 +489,7 @@ task:
# Use larger ccache cache, as this task compiles with multiple compilers /
# flag combinations
- CCACHE_MAXSIZE: "1GB"
+ CCACHE_MAXSIZE: "1500MB"
CCACHE_DIR: "/tmp/ccache_dir"
LINUX_CONFIGURE_FEATURES: *LINUX_CONFIGURE_FEATURES
@@ -524,7 +537,9 @@ task:
${LINUX_CONFIGURE_FEATURES} \
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
make -s -j${BUILD_JOBS} clean
+ ccache --zero-stats
time make -s -j${BUILD_JOBS} world-bin
+ ccache --show-stats
# gcc, cassert on, dtrace off
always:
@@ -535,7 +550,9 @@ task:
${LINUX_CONFIGURE_FEATURES} \
CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
make -s -j${BUILD_JOBS} clean
+ ccache --zero-stats
time make -s -j${BUILD_JOBS} world-bin
+ ccache --show-stats
# clang, cassert off, dtrace off
always:
@@ -545,7 +562,9 @@ task:
${LINUX_CONFIGURE_FEATURES} \
CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
make -s -j${BUILD_JOBS} clean
+ ccache --zero-stats
time make -s -j${BUILD_JOBS} world-bin
+ ccache --show-stats
# clang, cassert on, dtrace on
always:
@@ -557,7 +576,9 @@ task:
${LINUX_CONFIGURE_FEATURES} \
CC="ccache clang" CXX="ccache clang++" CLANG="ccache clang"
make -s -j${BUILD_JOBS} clean
+ ccache --zero-stats
time make -s -j${BUILD_JOBS} world-bin
+ ccache --show-stats
# cross-compile to windows
always:
@@ -568,7 +589,9 @@ task:
CC="ccache x86_64-w64-mingw32-gcc" \
CXX="ccache x86_64-w64-mingw32-g++"
make -s -j${BUILD_JOBS} clean
+ ccache --zero-stats
time make -s -j${BUILD_JOBS} world-bin
+ ccache --show-stats
###
# Verify docs can be built
--
2.17.1
>From 01e9abd386a4e6cc0125b97617fb42e695898cbf Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Tue, 26 Jul 2022 20:30:02 -0500
Subject: [PATCH 04/25] cirrus/ccache: add explicit cache keys..
Since otherwise, building with ci-os-only will probably fail to use the normal
cache, since the cache key is computed using both the task name and its *index*
in the list of caches (internal/executor/cache.go:184).
---
.cirrus.yml | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/.cirrus.yml b/.cirrus.yml
index c0616e8560d..4af6105aac7 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -68,6 +68,9 @@ task:
ccache_cache:
folder: $CCACHE_DIR
+ fingerprint_key: ccache/freebsd
+ reupload_on_changes: true
+
# Workaround around performance issues due to 32KB block size
repartition_script: src/tools/ci/gcp_freebsd_repartition.sh
create_user_script: |
@@ -175,6 +178,8 @@ task:
ccache_cache:
folder: ${CCACHE_DIR}
+ fingerprint_key: ccache/linux
+ reupload_on_changes: true
sysinfo_script: |
id
@@ -289,6 +294,9 @@ task:
ccache_cache:
folder: $CCACHE_DIR
+ fingerprint_key: ccache/macos
+ reupload_on_changes: true
+
configure_script: |
brewpath="/usr/local"
INCLUDES="${brewpath}/include:${INCLUDES}"
@@ -513,6 +521,8 @@ task:
ccache_cache:
folder: $CCACHE_DIR
+ fingerprint_key: ccache/warnings
+ reupload_on_changes: true
setup_additional_packages_script: |
#apt-get update
--
2.17.1
>From 8de5c977686270b0a4e666a924ebe820a245913a Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sun, 24 Jul 2022 23:09:12 -0500
Subject: [PATCH 05/25] silence make distprep and generated-headers
this saves vertical screen space.
https://www.postgresql.org/message-id/20220221164736.rq3ornzjdkmwk...@alap3.anarazel.de
ci-os-only: linux
---
src/backend/Makefile | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/src/backend/Makefile b/src/backend/Makefile
index f498cfd5930..f4a5d4a4aa6 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -128,15 +128,15 @@ storage/lmgr/lwlocknames.h: storage/lmgr/generate-lwlocknames.pl storage/lmgr/lw
# run this unconditionally to avoid needing to know its dependencies here:
submake-catalog-headers:
- $(MAKE) -C catalog distprep generated-header-symlinks
+ @$(MAKE) -s -C catalog distprep generated-header-symlinks
# run this unconditionally to avoid needing to know its dependencies here:
submake-nodes-headers:
- $(MAKE) -C nodes distprep generated-header-symlinks
+ @$(MAKE) -s -C nodes distprep generated-header-symlinks
# run this unconditionally to avoid needing to know its dependencies here:
submake-utils-headers:
- $(MAKE) -C utils distprep generated-header-symlinks
+ @$(MAKE) -s -C utils distprep generated-header-symlinks
.PHONY: submake-catalog-headers submake-nodes-headers submake-utils-headers
@@ -172,14 +172,15 @@ utils/probes.o: utils/probes.d $(SUBDIROBJS)
##########################################################################
# Be sure that these files get removed by the maintainer-clean target
+#
distprep:
$(MAKE) -C parser gram.c gram.h scan.c
$(MAKE) -C bootstrap bootparse.c bootscanner.c
$(MAKE) -C catalog distprep
- $(MAKE) -C nodes distprep
+ $(MAKE) -C nodes distprep
$(MAKE) -C replication repl_gram.c repl_scanner.c syncrep_gram.c syncrep_scanner.c
$(MAKE) -C storage/lmgr lwlocknames.h lwlocknames.c
- $(MAKE) -C utils distprep
+ $(MAKE) -C utils distprep
$(MAKE) -C utils/adt jsonpath_gram.c jsonpath_scan.c
$(MAKE) -C utils/misc guc-file.c
--
2.17.1
>From eaf263ccaa8310c5d9834b97e93ad8434d63296e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sun, 24 Jul 2022 22:44:53 -0500
Subject: [PATCH 06/25] pg_regress: run more quietly
The number of lines of output should be closer to 1 per test, rather than 25 +
1 per test.
https://www.postgresql.org/message-id/flat/20220221173109.yl6dqqu3ud52ripd%40alap3.anarazel.de
---
src/test/regress/pg_regress.c | 27 +++++----------------------
1 file changed, 5 insertions(+), 22 deletions(-)
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index a803355f8ee..9e05f15ff75 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -214,7 +214,7 @@ header(const char *fmt,...)
vsnprintf(tmp, sizeof(tmp), fmt, ap);
va_end(ap);
- fprintf(stdout, "============== %-38s ==============\n", tmp);
+ fprintf(stdout, "============== %-48s ==============\n", tmp);
fflush(stdout);
}
@@ -744,6 +744,7 @@ initialize_environment(void)
pghost = "localhost"; /* DefaultHost in fe-connect.c */
}
+ // what about these ?
if (pghost && pgport)
printf(_("(using postmaster on %s, port %s)\n"), pghost, pgport);
if (pghost && !pgport)
@@ -977,7 +978,7 @@ psql_start_command(void)
StringInfo buf = makeStringInfo();
appendStringInfo(buf,
- "\"%s%spsql\" -X",
+ "\"%s%spsql\" -X -q",
bindir ? bindir : "",
bindir ? "/" : "");
return buf;
@@ -1888,7 +1889,6 @@ drop_database_if_exists(const char *dbname)
{
StringInfo buf = psql_start_command();
- header(_("dropping database \"%s\""), dbname);
/* Set warning level so we don't see chatter about nonexistent DB */
psql_add_command(buf, "SET client_min_messages = warning");
psql_add_command(buf, "DROP DATABASE IF EXISTS \"%s\"", dbname);
@@ -1905,7 +1905,6 @@ create_database(const char *dbname)
* We use template0 so that any installation-local cruft in template1 will
* not mess up the tests.
*/
- header(_("creating database \"%s\""), dbname);
if (encoding)
psql_add_command(buf, "CREATE DATABASE \"%s\" TEMPLATE=template0 ENCODING='%s'%s", dbname, encoding,
(nolocale) ? " LC_COLLATE='C' LC_CTYPE='C'" : "");
@@ -1928,7 +1927,6 @@ create_database(const char *dbname)
*/
for (sl = loadextension; sl != NULL; sl = sl->next)
{
- header(_("installing %s"), sl->str);
psql_command(dbname, "CREATE EXTENSION IF NOT EXISTS \"%s\"", sl->str);
}
}
@@ -1938,7 +1936,6 @@ drop_role_if_exists(const char *rolename)
{
StringInfo buf = psql_start_command();
- header(_("dropping role \"%s\""), rolename);
/* Set warning level so we don't see chatter about nonexistent role */
psql_add_command(buf, "SET client_min_messages = warning");
psql_add_command(buf, "DROP ROLE IF EXISTS \"%s\"", rolename);
@@ -1950,7 +1947,6 @@ create_role(const char *rolename, const _stringlist *granted_dbs)
{
StringInfo buf = psql_start_command();
- header(_("creating role \"%s\""), rolename);
psql_add_command(buf, "CREATE ROLE \"%s\" WITH LOGIN", rolename);
for (; granted_dbs != NULL; granted_dbs = granted_dbs->next)
{
@@ -2235,7 +2231,6 @@ regression_main(int argc, char *argv[],
if (directory_exists(temp_instance))
{
- header(_("removing existing temp instance"));
if (!rmtree(temp_instance, true))
{
fprintf(stderr, _("\n%s: could not remove temp instance \"%s\"\n"),
@@ -2244,8 +2239,6 @@ regression_main(int argc, char *argv[],
}
}
- header(_("creating temporary instance"));
-
/* make the temp instance top directory */
make_directory(temp_instance);
@@ -2255,7 +2248,6 @@ regression_main(int argc, char *argv[],
make_directory(buf);
/* initdb */
- header(_("initializing database system"));
snprintf(buf, sizeof(buf),
"\"%s%sinitdb\" -D \"%s/data\" --no-clean --no-sync%s%s > \"%s/log/initdb.log\" 2>&1",
bindir ? bindir : "",
@@ -2360,7 +2352,6 @@ regression_main(int argc, char *argv[],
/*
* Start the temp postmaster
*/
- header(_("starting postmaster"));
snprintf(buf, sizeof(buf),
"\"%s%spostgres\" -D \"%s/data\" -F%s "
"-c \"listen_addresses=%s\" -k \"%s\" "
@@ -2449,7 +2440,7 @@ regression_main(int argc, char *argv[],
#else
#define ULONGPID(x) (unsigned long) (x)
#endif
- printf(_("running on port %d with PID %lu\n"),
+ header(_("started postmaster on port %d with PID %lu"),
port, ULONGPID(postmaster_pid));
}
else
@@ -2481,7 +2472,6 @@ regression_main(int argc, char *argv[],
/*
* Ready to run the tests
*/
- header(_("running regression test queries"));
for (sl = schedulelist; sl != NULL; sl = sl->next)
{
@@ -2509,7 +2499,6 @@ regression_main(int argc, char *argv[],
*/
if (temp_instance && fail_count == 0 && fail_ignore_count == 0)
{
- header(_("removing temporary instance"));
if (!rmtree(temp_instance, true))
fprintf(stderr, _("\n%s: could not remove temp instance \"%s\"\n"),
progname, temp_instance);
@@ -2543,13 +2532,7 @@ regression_main(int argc, char *argv[],
success_count + fail_count + fail_ignore_count,
fail_ignore_count);
- putchar('\n');
- for (i = strlen(buf); i > 0; i--)
- putchar('=');
- printf("\n%s\n", buf);
- for (i = strlen(buf); i > 0; i--)
- putchar('=');
- putchar('\n');
+ header("%s", buf);
putchar('\n');
if (file_size(difffilename) > 0)
--
2.17.1
>From af8ec5077c6f87efcc012cd6b849ae1054d71ae1 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Thu, 14 Apr 2022 06:27:07 -0500
Subject: [PATCH 07/25] cirrus: enable various runtime checks on macos and
freebsd
windows is slower than freebsd and mac, so it's okay to enable options which
will slow them down some. Also, the cirrusci mac instances always have lot of
cores available.
See:
https://www.postgresql.org/message-id/20211217193159.pwrelhiyx7kev...@alap3.anarazel.de
https://www.postgresql.org/message-id/20211213211223.vkgg3wwiss2tragj%40alap3.anarazel.de
https://www.postgresql.org/message-id/CAH2-WzmevBhKNEtqX3N-Tkb0gVBHH62C0KfeTxXzqYES_PiFiA%40mail.gmail.com
https://www.postgresql.org/message-id/20220325000933.vgazz7pjk2ytj...@alap3.anarazel.de
ci-os-only: freebsd, macos
---
.cirrus.yml | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 4af6105aac7..ea6837d94f9 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -113,7 +113,9 @@ task:
\
CC="ccache cc" \
CXX="ccache c++" \
- CFLAGS="-Og -ggdb"
+ CPPFLAGS="-DRELCACHE_FORCE_RELEASE -DCOPY_PARSE_PLAN_TREES -DWRITE_READ_PARSE_PLAN_TREES -DRAW_EXPRESSION_COVERAGE_TEST" \
+ CXXFLAGS="-Og -ggdb -march=native -mtune=native" \
+ CFLAGS="-Og -ggdb -march=native -mtune=native"
EOF
build_script: |
su postgres -c "ccache --zero-stats"
@@ -336,8 +338,8 @@ task:
CC="ccache cc" \
CXX="ccache c++" \
CLANG="ccache ${brewpath}/llvm/bin/ccache" \
- CFLAGS="-Og -ggdb" \
- CXXFLAGS="-Og -ggdb" \
+ CFLAGS="-Og -ggdb -DRANDOMIZE_ALLOCATED_MEMORY" \
+ CXXFLAGS="-Og -ggdb -DRANDOMIZE_ALLOCATED_MEMORY" \
\
LLVM_CONFIG=${brewpath}/llvm/bin/llvm-config \
PYTHON=python3
--
2.17.1
>From 6a6a97fc869fd1fd8b7ab5da5147f145581634f9 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Fri, 24 Jun 2022 00:09:12 -0500
Subject: [PATCH 08/25] cirrus/freebsd: run with more CPUs+RAM and do not
repartitiion
There was some historic problem where tests under freebsd took 8+ minutes (and
before 4a288a37f took 15 minutes).
This reduces test time from 10min to 3min.
4 CPUs 4 tests https://cirrus-ci.com/task/4880240739614720
4 CPUs 6 tests https://cirrus-ci.com/task/4664440120410112 https://cirrus-ci.com/task/4586784884523008
4 CPUs 8 tests https://cirrus-ci.com/task/5001995491737600
6 CPUs https://cirrus-ci.com/task/6678321684545536
8 CPUs https://cirrus-ci.com/task/6264854121021440
See also:
https://www.postgresql.org/message-id/flat/20220310033347.hgxk4pyarzq4h...@alap3.anarazel.de#f36c0b17e33e31e7925e7e5812998686
8 jobs 7min https://cirrus-ci.com/task/6186376667332608
xi-os-only: freebsd
---
.cirrus.yml | 12 ++++--------
1 file changed, 4 insertions(+), 8 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index ea6837d94f9..38cc25fcf7c 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -42,11 +42,9 @@ task:
name: FreeBSD - 13
env:
- # FreeBSD on GCP is slow when running with larger number of CPUS /
- # jobs. Using one more job than cpus seems to work best.
- CPUS: 2
- BUILD_JOBS: 3
- TEST_JOBS: 3
+ CPUS: 4
+ BUILD_JOBS: 4
+ TEST_JOBS: 6
CCACHE_DIR: /tmp/ccache_dir
@@ -57,7 +55,7 @@ task:
image: family/pg-ci-freebsd-13
platform: freebsd
cpu: $CPUS
- memory: 2G
+ memory: 3G
disk: 50
sysinfo_script: |
@@ -71,8 +69,6 @@ task:
fingerprint_key: ccache/freebsd
reupload_on_changes: true
- # Workaround around performance issues due to 32KB block size
- repartition_script: src/tools/ci/gcp_freebsd_repartition.sh
create_user_script: |
pw useradd postgres
chown -R postgres:postgres .
--
2.17.1
>From fd1c36a0bd8fa608ccdff5be3735dac5e3e48bf3 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Wed, 27 Jul 2022 16:54:47 -0500
Subject: [PATCH 09/25] cirrus/freebsd: run build+check in a make vpath
ci-os-only: freebsd
---
.cirrus.yml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 38cc25fcf7c..65a7a3774ff 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -86,7 +86,7 @@ task:
# freebsd already takes longer than other platforms except for windows.
configure_script: |
su postgres <<-EOF
- ./configure \
+ mkdir build; cd build; ../configure \
--enable-cassert --enable-debug --enable-tap-tests \
--enable-nls \
\
@@ -115,7 +115,7 @@ task:
EOF
build_script: |
su postgres -c "ccache --zero-stats"
- su postgres -c "gmake -s -j${BUILD_JOBS} world-bin"
+ su postgres -c "cd build && gmake -s -j${BUILD_JOBS} world-bin"
su postgres -c "ccache --show-stats"
upload_caches: ccache
@@ -124,7 +124,7 @@ task:
# https://savannah.gnu.org/bugs/?60774
# script uses pseudo-ttys, which do support locking.
test_world_script:
- - su postgres -c "time script test.log gmake -s -j${TEST_JOBS} ${CHECK} ${CHECKFLAGS}"
+ - su postgres -c "cd build && time script test.log gmake -s -j${TEST_JOBS} ${CHECK} ${CHECKFLAGS}"
on_failure:
<<: *on_failure
--
2.17.1
>From 7052a32a21752b59632225684fc9426bb94e46e0 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sun, 13 Feb 2022 17:56:40 -0600
Subject: [PATCH 10/25] cirrus/windows: increase timeout to 25min
---
.cirrus.yml | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 65a7a3774ff..54124d0fe34 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -395,9 +395,9 @@ task:
# If tests hang forever, cirrus eventually times out. In that case log
# output etc is not uploaded, making the problem hard to debug. Of course
# tests internally should have shorter timeouts, but that's proven to not
- # be sufficient. 15min currently is fast enough to finish individual test
+ # be sufficient. 25min currently is fast enough to finish individual test
# "suites".
- T_C: "\"C:/Program Files/Git/usr/bin/timeout.exe\" -v -k60s 15m"
+ T_C: "\"C:/Program Files/Git/usr/bin/timeout.exe\" -v -k60s 25m"
# startcreate_script starts a postgres instance that we don't want to get
# killed at the end of that script (it's stopped in stop_script). Can't
--
2.17.1
>From 77d084cd920a790c307a009c009808fa7035ad2d Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Fri, 25 Feb 2022 17:00:33 -0600
Subject: [PATCH 11/25] vcregress: add alltaptests
https://www.postgresql.org/message-id/20220326031353.gg28...@telsasoft.com
https://www.postgresql.org/message-id/flat/87a81b91-87bf-c0bc-7e4f-06dffadcf...@dunslane.net
In passing, document taptest PROVE_FLAGS
ci-os-only: windows
See also:
d835dd6685246f0737ca42ab68242210681bb220
13d856e177e69083f543d6383eeda9e12ce3c55c
fed6df486dca1b9e53d3f560031b9a236c99f4bb
---
.cirrus.yml | 10 ++-------
src/tools/msvc/vcregress.pl | 43 +++++++++++++++++++++++++++++++++----
2 files changed, 41 insertions(+), 12 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 54124d0fe34..f19558e67e0 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -455,14 +455,8 @@ task:
test_ssl_script: |
set with_ssl=openssl
%T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/ssl/
- test_subscription_script: |
- %T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/subscription/
- test_authentication_script: |
- %T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/authentication/
- test_recovery_script: |
- %T_C% perl src/tools/msvc/vcregress.pl recoverycheck
- test_bin_script: |
- %T_C% perl src/tools/msvc/vcregress.pl bincheck
+ test_tap_script: |
+ %T_C% perl src/tools/msvc/vcregress.pl alltaptests
test_ecpg_script: |
rem tries to build additional stuff
vcvarsall x64
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 2d6ccd45419..6495f33e593 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -51,7 +51,7 @@ if (-e "src/tools/msvc/buildenv.pl")
my $what = shift || "";
if ($what =~
- /^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck|taptest)$/i
+ /^(check|installcheck|plcheck|contribcheck|modulescheck|ecpgcheck|isolationcheck|upgradecheck|bincheck|recoverycheck|taptest|alltaptests)$/i
)
{
$what = uc $what;
@@ -109,6 +109,7 @@ my %command = (
BINCHECK => \&bincheck,
RECOVERYCHECK => \&recoverycheck,
UPGRADECHECK => \&upgradecheck, # no-op
+ ALLTAPTESTS => \&alltaptests,
TAPTEST => \&taptest,);
my $proc = $command{$what};
@@ -296,6 +297,9 @@ sub tap_check
# add the module build dir as the second element in the PATH
$ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
+ print "============================================================\n";
+ print "Checking $dir: @args\n";
+
rmtree('tmp_check');
system(@args);
my $status = $? >> 8;
@@ -310,8 +314,7 @@ sub bincheck
my $mstat = 0;
- # Find out all the existing TAP tests by looking for t/ directories
- # in the tree.
+ # Find the TAP tests by looking for t/ directories
my @bin_dirs = glob("$topdir/src/bin/*");
# Process each test
@@ -326,6 +329,36 @@ sub bincheck
return;
}
+sub alltaptests
+{
+ InstallTemp();
+
+ my $mstat = 0;
+
+ # Find out all the existing TAP tests by looking for t/ directories
+ # in the tree.
+ my @tap_dirs = ();
+ my @top_dir = ($topdir);
+ File::Find::find(
+ { wanted => sub {
+ /^t\z/s
+ && $File::Find::name !~ /\/(kerberos|ldap|ssl|ssl_passphrase_callback)\// # opt-in: warn about these?
+ && push(@tap_dirs, $File::Find::name);
+ }
+ },
+ @top_dir);
+
+ # Process each test
+ foreach my $test_path (@tap_dirs)
+ {
+ my $dir = dirname($test_path);
+ my $status = tap_check($dir);
+ $mstat ||= $status;
+ }
+ exit $mstat if $mstat;
+ return;
+}
+
sub taptest
{
my $dir = shift;
@@ -676,6 +709,7 @@ sub usage
print STDERR
"Usage: vcregress.pl <mode> [<arg>]\n\n",
"Options for <mode>:\n",
+ " alltaptests run all tap tests (except kerberos, ldap, ssl, ssl_passphrase_callback)\n",
" bincheck run tests of utilities in src/bin/\n",
" check deploy instance and run regression tests on it\n",
" contribcheck run tests of modules in contrib/\n",
@@ -693,6 +727,7 @@ sub usage
"\nOptions for <arg>: (used by contribcheck and modulescheck)\n",
" install also run tests which require a new instance\n",
"\nOption for <arg>: for taptest\n",
- " TEST_DIR (required) directory where tests reside\n";
+ " TEST_DIR (required) directory where tests reside\n",
+ " PROVE_FLAGS flags to pass to prove\n";
exit(1);
}
--
2.17.1
>From 90b20320887dce46e11bb542cc96b6e54f0189bf Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Thu, 31 Mar 2022 23:52:10 -0500
Subject: [PATCH 12/25] tmp: run tap tests first
ci-os-only: windows
---
.cirrus.yml | 7 +++++--
1 file changed, 5 insertions(+), 2 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index f19558e67e0..f6763caecc4 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -434,6 +434,11 @@ task:
# Installation on windows currently only completely works from src/tools/msvc
- cd src/tools/msvc && perl install.pl %CIRRUS_WORKING_DIR%/tmp_install
+ this_tap_script: |
+ %T_C% perl src/tools/msvc/vcregress.pl taptest src/interfaces/libpq
+ test_tap_script: |
+ %T_C% perl src/tools/msvc/vcregress.pl alltaptests
+
test_regress_parallel_script: |
%T_C% perl src/tools/msvc/vcregress.pl check parallel
startcreate_script: |
@@ -455,8 +460,6 @@ task:
test_ssl_script: |
set with_ssl=openssl
%T_C% perl src/tools/msvc/vcregress.pl taptest ./src/test/ssl/
- test_tap_script: |
- %T_C% perl src/tools/msvc/vcregress.pl alltaptests
test_ecpg_script: |
rem tries to build additional stuff
vcvarsall x64
--
2.17.1
>From 2e6afdc5958c311c992e99a6d205b8bc01ee08cb Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sat, 19 Feb 2022 13:06:52 -0600
Subject: [PATCH 13/25] set TESTDIR from src/test/perl rather than
Makefile/vcregress
https://www.postgresql.org/message-id/flat/20220219234148.GC9008%40telsasoft.com
These seem most likely to break:
make check -C src/bin/psql
make check -C src/bin/pgbench
make check -C src/test/modules/test_misc
make check -C src/test/modules/libpq_pipeline
PROVE_TESTS=t/027_stream_regress.pl make check -C src/test/recovery
---
src/Makefile.global.in | 9 ++++++---
src/test/perl/PostgreSQL/Test/Utils.pm | 16 +++++++++-------
src/tools/msvc/vcregress.pl | 1 -
3 files changed, 15 insertions(+), 11 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index 5664c645f82..e48b12edfac 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -453,8 +453,9 @@ echo "+++ tap install-check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
@@ -464,8 +465,9 @@ echo "+++ tap install-check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$(CURDIR):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(top_builddir)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
@@ -476,7 +478,8 @@ echo "+++ tap check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
cd $(srcdir) && \
- TESTDIR='$(CURDIR)' $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
+ $(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
+ PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
$(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
endef
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index 1ca2cc59170..063a19b1df5 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -189,19 +189,21 @@ INIT
# test may still fail, but it's more likely to report useful facts.
$SIG{PIPE} = 'IGNORE';
- # Determine output directories, and create them. The base path is the
- # TESTDIR environment variable, which is normally set by the invoking
- # Makefile.
- $tmp_check = $ENV{TESTDIR} ? "$ENV{TESTDIR}/tmp_check" : "tmp_check";
+ my $test_dir = File::Spec->rel2abs($ENV{PG_SUBDIR} || dirname(dirname($0)));
+ my $test_name = basename($0);
+ $test_name =~ s/\.[^.]+$//;
+
+ $ENV{TESTDIR} = $test_dir;
+
+ # Determine output directories, and create them.
+ $tmp_check = "$test_dir/tmp_check";
$log_path = "$tmp_check/log";
mkdir $tmp_check;
mkdir $log_path;
# Open the test log file, whose name depends on the test name.
- $test_logfile = basename($0);
- $test_logfile =~ s/\.[^.]+$//;
- $test_logfile = "$log_path/regress_log_$test_logfile";
+ $test_logfile = "$log_path/regress_log_$test_name";
open my $testlog, '>', $test_logfile
or die "could not open STDOUT to logfile \"$test_logfile\": $!";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 6495f33e593..822cbcdef47 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -292,7 +292,6 @@ sub tap_check
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
$ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
- $ENV{TESTDIR} = "$dir";
my $module = basename $dir;
# add the module build dir as the second element in the PATH
$ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
--
2.17.1
>From 949a6599ab1feb3c1dd31dfa5c3279da0db86e80 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Fri, 25 Feb 2022 17:23:26 -0600
Subject: [PATCH 14/25] s!also set PATH.
This essentially reverts commit f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
(See also 795862c280c5949bafcd8c44543d887fd32b590a).
NO And partially reverts 6b04abdfc5e0653542ac5d586e639185a8c61a39
XXX: also set PATH=bindir ?
linux, windows
ci-os-only: freebsd,
---
src/Makefile.global.in | 6 +++---
src/test/perl/PostgreSQL/Test/Utils.pm | 10 ++++++++++
src/tools/msvc/vcregress.pl | 4 ----
3 files changed, 13 insertions(+), 7 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index e48b12edfac..ac970215495 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -441,7 +441,7 @@ ld_library_path_var = LD_LIBRARY_PATH
# need something more here. If not defined then the expansion does
# nothing.
with_temp_install = \
- PATH="$(abs_top_builddir)/tmp_install$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(abs_top_builddir)/tmp_install$(bindir):$$PATH" \
$(call add_to_path,$(strip $(ld_library_path_var)),$(abs_top_builddir)/tmp_install$(libdir)) \
$(with_temp_install_extra)
@@ -453,7 +453,7 @@ echo "+++ tap install-check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
cd $(srcdir) && \
- PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
@@ -465,7 +465,7 @@ echo "+++ tap install-check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
cd $(srcdir) && \
- PATH="$(bindir):$(CURDIR):$$PATH" \
+ PATH="$(bindir):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(top_builddir)' \
PG_SUBDIR='$(CURDIR)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index 063a19b1df5..4a1629a8e16 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -195,6 +195,16 @@ INIT
$ENV{TESTDIR} = $test_dir;
+ if ($PostgreSQL::Test::Utils::windows_os &&
+ $Config{osname} eq 'MSWin32')
+ {
+ $ENV{PATH} =~ s!;!;$test_dir;$test_dir/test;!;
+ }
+ else
+ {
+ $ENV{PATH} =~ s!:!:$test_dir:$test_dir/test:!;
+ }
+
# Determine output directories, and create them.
$tmp_check = "$test_dir/tmp_check";
$log_path = "$tmp_check/log";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 822cbcdef47..5dcc3419ad7 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -292,10 +292,6 @@ sub tap_check
$ENV{PG_REGRESS} = "$topdir/$Config/pg_regress/pg_regress";
$ENV{REGRESS_SHLIB} = "$topdir/src/test/regress/regress.dll";
- my $module = basename $dir;
- # add the module build dir as the second element in the PATH
- $ENV{PATH} =~ s!;!;$topdir/$Config/$module;!;
-
print "============================================================\n";
print "Checking $dir: @args\n";
--
2.17.1
>From 602983b2cf37fc43465c62330b2e15e9d6d2035d Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Fri, 26 Aug 2022 12:00:10 -0500
Subject: [PATCH 15/25] f!and chdir
compare: 1053f85 91ae677 557eb01
sending the srcdir means that libpq_pipeline fails; it needs to set path to the
build dir (testdir). Why doesn't it break the other tests that set PATH, though ??
pipeline is working now, but failing in basebackup test??
https://github.com/justinpryzby/postgres/runs/8044385257
XX-os-only: windows
ci-os-only: freebsd
---
src/Makefile.global.in | 15 ++++++---------
src/test/perl/PostgreSQL/Test/Utils.pm | 12 ++++++++++--
2 files changed, 16 insertions(+), 11 deletions(-)
diff --git a/src/Makefile.global.in b/src/Makefile.global.in
index ac970215495..b65a4630228 100644
--- a/src/Makefile.global.in
+++ b/src/Makefile.global.in
@@ -452,24 +452,22 @@ define prove_installcheck
echo "+++ tap install-check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
-cd $(srcdir) && \
PATH="$(bindir):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(CURDIR)/$(top_builddir)' \
- PG_SUBDIR='$(CURDIR)' \
+ PG_SRCDIR='$(srcdir)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
- $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+ $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),$(srcdir)/t/*.pl)
endef
else # PGXS case
define prove_installcheck
echo "+++ tap install-check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
-cd $(srcdir) && \
PATH="$(bindir):$$PATH" \
PGPORT='6$(DEF_PGPORT)' top_builddir='$(top_builddir)' \
- PG_SUBDIR='$(CURDIR)' \
+ PG_SRCDIR='$(srcdir)' \
PG_REGRESS='$(top_builddir)/src/test/regress/pg_regress' \
- $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+ $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),$(srcdir)/t/*.pl)
endef
endif # PGXS
@@ -477,11 +475,10 @@ define prove_check
echo "+++ tap check in $(subdir) +++" && \
rm -rf '$(CURDIR)'/tmp_check && \
$(MKDIR_P) '$(CURDIR)'/tmp_check && \
-cd $(srcdir) && \
$(with_temp_install) PGPORT='6$(DEF_PGPORT)' \
- PG_SUBDIR='$(CURDIR)' \
+ PG_SRCDIR='$(srcdir)' \
PG_REGRESS='$(CURDIR)/$(top_builddir)/src/test/regress/pg_regress' \
- $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),t/*.pl)
+ $(PROVE) $(PG_PROVE_FLAGS) $(PROVE_FLAGS) $(if $(PROVE_TESTS),$(PROVE_TESTS),$(srcdir)/t/*.pl)
endef
else
diff --git a/src/test/perl/PostgreSQL/Test/Utils.pm b/src/test/perl/PostgreSQL/Test/Utils.pm
index 4a1629a8e16..0b52101e835 100644
--- a/src/test/perl/PostgreSQL/Test/Utils.pm
+++ b/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -47,6 +47,7 @@ use warnings;
use Carp;
use Config;
use Cwd;
+use Cwd qw(abs_path);
use Exporter 'import';
use Fcntl qw(:mode :seek);
use File::Basename;
@@ -189,20 +190,27 @@ INIT
# test may still fail, but it's more likely to report useful facts.
$SIG{PIPE} = 'IGNORE';
- my $test_dir = File::Spec->rel2abs($ENV{PG_SUBDIR} || dirname(dirname($0)));
+ my $test_dir = abs_path($ENV{PG_SRCDIR} || dirname(dirname($0)));
+ #my $test_dir = File::Spec->rel2abs($ENV{PG_SUBDIR} || dirname(dirname($0)));
my $test_name = basename($0);
$test_name =~ s/\.[^.]+$//;
$ENV{TESTDIR} = $test_dir;
+ my $cwd = getcwd();
+ chdir($test_dir);
+ #$ENV{BUILD_DIR} = $cwd;
if ($PostgreSQL::Test::Utils::windows_os &&
$Config{osname} eq 'MSWin32')
{
+ # $test_dir is what's needed for windows
+ #$ENV{PATH} =~ s!;!;$test_dir;$test_dir/test;$cwd;$cwd/test;!;
$ENV{PATH} =~ s!;!;$test_dir;$test_dir/test;!;
}
else
{
- $ENV{PATH} =~ s!:!:$test_dir:$test_dir/test:!;
+ # $cwd is what's needed for a vpath build (and same as testdir for non-vpath build)
+ $ENV{PATH} =~ s!:!:$cwd:$cwd/test:!;
}
# Determine output directories, and create them.
--
2.17.1
>From c7396305c800aa9e218a47edc54c3371bf22f546 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sat, 19 Feb 2022 13:41:36 -0600
Subject: [PATCH 16/25] vcregress: run alltaptests in parallel
XX-os-only: windows
The test changes are needed to avoid these failures:
https://github.com/justinpryzby/postgres/runs/5174636590
[15:59:59.408] Bailout called. Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
[15:59:59.408] FAILED--Further testing stopped: could not create test directory "c:/cirrus/tmp_check/t_C:\cirrus\src\bin\pgbench\t\002_pgbench_no_server_stuff": Invalid argument
https://github.com/justinpryzby/postgres/runs/5174788205
[16:37:09.891] # Failed test 'reading traces/disallowed_in_pipeline.trace: could not open "traces/disallowed_in_pipeline.trace": The system cannot find the path specified at C:\cirrus\src\test\modules\libpq_pipeline\t\001_libpq_pipeline.pl line 70.
https://github.com/justinpryzby/postgres/runs/5174877506
pg_regress: could not open file "../regress/parallel_schedule" for reading: No such file or directory
XXX: avoid breaking src/bin/psql/t/010_tab_completion.pl:
See also:
f4ce6c4d3a30ec3a12c7f64b90a6fc82887ddd7b
795862c280c5949bafcd8c44543d887fd32b590a
db973ffb3ca43e65a0bf15175a35184a53bf977d
---
src/bin/pgbench/t/002_pgbench_no_server.pl | 4 ++--
src/tools/msvc/vcregress.pl | 14 ++++----------
2 files changed, 6 insertions(+), 12 deletions(-)
diff --git a/src/bin/pgbench/t/002_pgbench_no_server.pl b/src/bin/pgbench/t/002_pgbench_no_server.pl
index 50bde7dd0fc..bbbb49bb23d 100644
--- a/src/bin/pgbench/t/002_pgbench_no_server.pl
+++ b/src/bin/pgbench/t/002_pgbench_no_server.pl
@@ -8,12 +8,12 @@
use strict;
use warnings;
+use File::Basename;
use PostgreSQL::Test::Utils;
use Test::More;
# create a directory for scripts
-my $testname = $0;
-$testname =~ s,.*/,,;
+my $testname = basename($0);
$testname =~ s/\.pl$//;
my $testdir = "$PostgreSQL::Test::Utils::tmp_check/t_${testname}_stuff";
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 5dcc3419ad7..49168e298ac 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -328,8 +328,6 @@ sub alltaptests
{
InstallTemp();
- my $mstat = 0;
-
# Find out all the existing TAP tests by looking for t/ directories
# in the tree.
my @tap_dirs = ();
@@ -343,14 +341,10 @@ sub alltaptests
},
@top_dir);
- # Process each test
- foreach my $test_path (@tap_dirs)
- {
- my $dir = dirname($test_path);
- my $status = tap_check($dir);
- $mstat ||= $status;
- }
- exit $mstat if $mstat;
+ # Run all the tap tests in a single prove instance for better performance
+ $ENV{PROVE_TESTS} = "@tap_dirs";
+ my $status = tap_check('PROVE_FLAGS=--ext=.pl', "$topdir");
+ exit $status if $status;
return;
}
--
2.17.1
>From 4380918b74661d5af3087816583e2503bf0f2c82 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Thu, 26 May 2022 22:24:41 -0500
Subject: [PATCH 17/25] another way to install uri_regress/testclient
This reverts commit a17fd67d2f2861ae0ce00d1aeefdf2facc47cd5e.
TODO: partially revert 6b04abdfc5e0653542ac5d586e639185a8c61a39
See also:
a17fd67d2f2861ae0ce00d1aeefdf2facc47cd5e
https://www.postgresql.org/message-id/20220416144454.GX26620%40telsasoft.com
https://www.postgresql.org/message-id/20220223211606.ikcwoolsz2nohsw6%40alap3.anarazel.de
TODO: doc/src/sgml/extend.sgml
ci-os-only: linux, windows
---
src/interfaces/libpq/test/Makefile | 8 +++----
src/tools/msvc/Mkvcbuild.pm | 36 +++++++++++++-----------------
2 files changed, 19 insertions(+), 25 deletions(-)
diff --git a/src/interfaces/libpq/test/Makefile b/src/interfaces/libpq/test/Makefile
index 75ac08f943d..91c8d87d528 100644
--- a/src/interfaces/libpq/test/Makefile
+++ b/src/interfaces/libpq/test/Makefile
@@ -14,11 +14,11 @@ endif
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += $(libpq_pgport)
-PROGS = libpq_testclient libpq_uri_regress
+PROGRAMS = libpq_testclient libpq_uri_regress
-all: $(PROGS)
+all: $(PROGRAMS)
-$(PROGS): $(WIN32RES)
+$(PROGRAMS): $(WIN32RES)
clean distclean maintainer-clean:
- rm -f $(PROGS) *.o
+ rm -f $(PROGRAMS) *.o
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index c3cef625ad3..f327e631cdd 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -36,11 +36,14 @@ my @unlink_on_exit;
# Set of variables for modules in contrib/ and src/test/modules/
my @contrib_uselibpq = ();
-my @contrib_uselibpgport = ();
+my @contrib_uselibpgport = ('libpq_uri_regress', 'libpq_testclient');
my @contrib_uselibpgcommon = ();
my $contrib_extralibs = { 'libpq_pipeline' => ['ws2_32.lib'] };
my $contrib_extraincludes = {};
-my $contrib_extrasource = {};
+my $contrib_extrasource = {
+ 'libpq_uri_regress' => ['src/interfaces/libpq/test/libpq_uri_regress.c'],
+ 'libpq_testclient' => ['src/interfaces/libpq/test/libpq_testclient.c'],
+};
my @contrib_excludes = (
'bool_plperl', 'commit_ts',
'hstore_plperl', 'hstore_plpython',
@@ -289,24 +292,6 @@ sub mkvcbuild
$libpqwalreceiver->AddIncludeDir('src/interfaces/libpq');
$libpqwalreceiver->AddReference($postgres, $libpq);
- my $libpq_testclient =
- $solution->AddProject('libpq_testclient', 'exe', 'misc',
- 'src/interfaces/libpq/test');
- $libpq_testclient->AddFile(
- 'src/interfaces/libpq/test/libpq_testclient.c');
- $libpq_testclient->AddIncludeDir('src/interfaces/libpq');
- $libpq_testclient->AddReference($libpgport, $libpq);
- $libpq_testclient->AddLibrary('ws2_32.lib');
-
- my $libpq_uri_regress =
- $solution->AddProject('libpq_uri_regress', 'exe', 'misc',
- 'src/interfaces/libpq/test');
- $libpq_uri_regress->AddFile(
- 'src/interfaces/libpq/test/libpq_uri_regress.c');
- $libpq_uri_regress->AddIncludeDir('src/interfaces/libpq');
- $libpq_uri_regress->AddReference($libpgport, $libpq);
- $libpq_uri_regress->AddLibrary('ws2_32.lib');
-
my $pgoutput = $solution->AddProject('pgoutput', 'dll', '',
'src/backend/replication/pgoutput');
$pgoutput->AddReference($postgres);
@@ -477,7 +462,7 @@ sub mkvcbuild
push @contrib_excludes, 'uuid-ossp';
}
- foreach my $subdir ('contrib', 'src/test/modules')
+ foreach my $subdir ('contrib', 'src/test/modules', 'src/interfaces/libpq')
{
opendir($D, $subdir) || croak "Could not opendir on $subdir!\n";
while (my $d = readdir($D))
@@ -988,6 +973,15 @@ sub AddContrib
AdjustContribProj($proj);
push @projects, $proj;
}
+ elsif ($mf =~ /^PROGRAMS\s*=\s*(.*)$/mg)
+ {
+ foreach my $proj (split /\s+/, $1)
+ {
+ my $p = $solution->AddProject($proj, 'exe', 'contrib', "$subdir/$n");
+ AdjustContribProj($p);
+ push @projects, $p;
+ }
+ }
else
{
croak "Could not determine contrib module type for $n\n";
--
2.17.1
>From b8d527abf77d1a8bc3390b8108e8cb7889b2fa77 Mon Sep 17 00:00:00 2001
From: Andres Freund <and...@anarazel.de>
Date: Thu, 24 Feb 2022 08:27:41 -0800
Subject: [PATCH 18/25] Move libpq_pipeline test into src/interfaces/libpq.
https://www.postgresql.org/message-id/20220227014626.vi7g7km554gmoape%40alap3.anarazel.de
windows,
ci-os-only: freebsd, linux
---
.../libpq/t/002_pipeline.pl} | 2 +-
src/interfaces/libpq/test/.gitignore | 1 +
src/interfaces/libpq/test/Makefile | 2 +-
.../libpq/test}/libpq_pipeline.c | 0
.../test}/traces/disallowed_in_pipeline.trace | 0
.../libpq/test}/traces/multi_pipelines.trace | 0
.../libpq/test}/traces/nosync.trace | 0
.../libpq/test}/traces/pipeline_abort.trace | 0
.../libpq/test}/traces/pipeline_idle.trace | 0
.../libpq/test}/traces/prepared.trace | 0
.../libpq/test}/traces/simple_pipeline.trace | 0
.../libpq/test}/traces/singlerow.trace | 0
.../libpq/test}/traces/transaction.trace | 0
src/test/modules/Makefile | 1 -
src/test/modules/libpq_pipeline/.gitignore | 5 ----
src/test/modules/libpq_pipeline/Makefile | 25 -------------------
src/test/modules/libpq_pipeline/README | 1 -
src/tools/msvc/Mkvcbuild.pm | 5 ++--
18 files changed, 6 insertions(+), 36 deletions(-)
rename src/{test/modules/libpq_pipeline/t/001_libpq_pipeline.pl => interfaces/libpq/t/002_pipeline.pl} (96%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/libpq_pipeline.c (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/disallowed_in_pipeline.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/multi_pipelines.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/nosync.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/pipeline_abort.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/pipeline_idle.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/prepared.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/simple_pipeline.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/singlerow.trace (100%)
rename src/{test/modules/libpq_pipeline => interfaces/libpq/test}/traces/transaction.trace (100%)
delete mode 100644 src/test/modules/libpq_pipeline/.gitignore
delete mode 100644 src/test/modules/libpq_pipeline/Makefile
delete mode 100644 src/test/modules/libpq_pipeline/README
diff --git a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl b/src/interfaces/libpq/t/002_pipeline.pl
similarity index 96%
rename from src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
rename to src/interfaces/libpq/t/002_pipeline.pl
index 0821329c8d3..5ddde3425cc 100644
--- a/src/test/modules/libpq_pipeline/t/001_libpq_pipeline.pl
+++ b/src/interfaces/libpq/t/002_pipeline.pl
@@ -50,7 +50,7 @@ for my $testname (@tests)
my $expected;
my $result;
- $expected = slurp_file_eval("traces/$testname.trace");
+ $expected = slurp_file_eval("test/traces/$testname.trace");
next unless $expected ne "";
$result = slurp_file_eval($traceout);
next unless $result ne "";
diff --git a/src/interfaces/libpq/test/.gitignore b/src/interfaces/libpq/test/.gitignore
index 6ba78adb678..5c1a281d679 100644
--- a/src/interfaces/libpq/test/.gitignore
+++ b/src/interfaces/libpq/test/.gitignore
@@ -1,2 +1,3 @@
/libpq_testclient
/libpq_uri_regress
+/libpq_pipeline
diff --git a/src/interfaces/libpq/test/Makefile b/src/interfaces/libpq/test/Makefile
index 91c8d87d528..bd459062434 100644
--- a/src/interfaces/libpq/test/Makefile
+++ b/src/interfaces/libpq/test/Makefile
@@ -14,7 +14,7 @@ endif
override CPPFLAGS := -I$(libpq_srcdir) $(CPPFLAGS)
LDFLAGS_INTERNAL += $(libpq_pgport)
-PROGRAMS = libpq_testclient libpq_uri_regress
+PROGRAMS = libpq_testclient libpq_uri_regress libpq_pipeline
all: $(PROGRAMS)
diff --git a/src/test/modules/libpq_pipeline/libpq_pipeline.c b/src/interfaces/libpq/test/libpq_pipeline.c
similarity index 100%
rename from src/test/modules/libpq_pipeline/libpq_pipeline.c
rename to src/interfaces/libpq/test/libpq_pipeline.c
diff --git a/src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace b/src/interfaces/libpq/test/traces/disallowed_in_pipeline.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/disallowed_in_pipeline.trace
rename to src/interfaces/libpq/test/traces/disallowed_in_pipeline.trace
diff --git a/src/test/modules/libpq_pipeline/traces/multi_pipelines.trace b/src/interfaces/libpq/test/traces/multi_pipelines.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/multi_pipelines.trace
rename to src/interfaces/libpq/test/traces/multi_pipelines.trace
diff --git a/src/test/modules/libpq_pipeline/traces/nosync.trace b/src/interfaces/libpq/test/traces/nosync.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/nosync.trace
rename to src/interfaces/libpq/test/traces/nosync.trace
diff --git a/src/test/modules/libpq_pipeline/traces/pipeline_abort.trace b/src/interfaces/libpq/test/traces/pipeline_abort.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/pipeline_abort.trace
rename to src/interfaces/libpq/test/traces/pipeline_abort.trace
diff --git a/src/test/modules/libpq_pipeline/traces/pipeline_idle.trace b/src/interfaces/libpq/test/traces/pipeline_idle.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/pipeline_idle.trace
rename to src/interfaces/libpq/test/traces/pipeline_idle.trace
diff --git a/src/test/modules/libpq_pipeline/traces/prepared.trace b/src/interfaces/libpq/test/traces/prepared.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/prepared.trace
rename to src/interfaces/libpq/test/traces/prepared.trace
diff --git a/src/test/modules/libpq_pipeline/traces/simple_pipeline.trace b/src/interfaces/libpq/test/traces/simple_pipeline.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/simple_pipeline.trace
rename to src/interfaces/libpq/test/traces/simple_pipeline.trace
diff --git a/src/test/modules/libpq_pipeline/traces/singlerow.trace b/src/interfaces/libpq/test/traces/singlerow.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/singlerow.trace
rename to src/interfaces/libpq/test/traces/singlerow.trace
diff --git a/src/test/modules/libpq_pipeline/traces/transaction.trace b/src/interfaces/libpq/test/traces/transaction.trace
similarity index 100%
rename from src/test/modules/libpq_pipeline/traces/transaction.trace
rename to src/interfaces/libpq/test/traces/transaction.trace
diff --git a/src/test/modules/Makefile b/src/test/modules/Makefile
index 6c31c8707c2..bb7a80748fe 100644
--- a/src/test/modules/Makefile
+++ b/src/test/modules/Makefile
@@ -10,7 +10,6 @@ SUBDIRS = \
delay_execution \
dummy_index_am \
dummy_seclabel \
- libpq_pipeline \
plsample \
snapshot_too_old \
spgist_name_ops \
diff --git a/src/test/modules/libpq_pipeline/.gitignore b/src/test/modules/libpq_pipeline/.gitignore
deleted file mode 100644
index 3a11e786b83..00000000000
--- a/src/test/modules/libpq_pipeline/.gitignore
+++ /dev/null
@@ -1,5 +0,0 @@
-# Generated subdirectories
-/log/
-/results/
-/tmp_check/
-/libpq_pipeline
diff --git a/src/test/modules/libpq_pipeline/Makefile b/src/test/modules/libpq_pipeline/Makefile
deleted file mode 100644
index 65acc3e997e..00000000000
--- a/src/test/modules/libpq_pipeline/Makefile
+++ /dev/null
@@ -1,25 +0,0 @@
-# src/test/modules/libpq_pipeline/Makefile
-
-PGFILEDESC = "libpq_pipeline - test program for pipeline execution"
-PGAPPICON = win32
-
-PROGRAM = libpq_pipeline
-OBJS = $(WIN32RES) libpq_pipeline.o
-
-NO_INSTALL = 1
-
-PG_CPPFLAGS = -I$(libpq_srcdir)
-PG_LIBS_INTERNAL += $(libpq_pgport)
-
-TAP_TESTS = 1
-
-ifdef USE_PGXS
-PG_CONFIG = pg_config
-PGXS := $(shell $(PG_CONFIG) --pgxs)
-include $(PGXS)
-else
-subdir = src/test/modules/libpq_pipeline
-top_builddir = ../../../..
-include $(top_builddir)/src/Makefile.global
-include $(top_srcdir)/contrib/contrib-global.mk
-endif
diff --git a/src/test/modules/libpq_pipeline/README b/src/test/modules/libpq_pipeline/README
deleted file mode 100644
index d8174dd579a..00000000000
--- a/src/test/modules/libpq_pipeline/README
+++ /dev/null
@@ -1 +0,0 @@
-Test programs and libraries for libpq
diff --git a/src/tools/msvc/Mkvcbuild.pm b/src/tools/msvc/Mkvcbuild.pm
index f327e631cdd..eac3f05cdbb 100644
--- a/src/tools/msvc/Mkvcbuild.pm
+++ b/src/tools/msvc/Mkvcbuild.pm
@@ -36,13 +36,14 @@ my @unlink_on_exit;
# Set of variables for modules in contrib/ and src/test/modules/
my @contrib_uselibpq = ();
-my @contrib_uselibpgport = ('libpq_uri_regress', 'libpq_testclient');
-my @contrib_uselibpgcommon = ();
+my @contrib_uselibpgport = ('libpq_uri_regress', 'libpq_testclient', 'libpq_pipeline');
+my @contrib_uselibpgcommon = ('libpq_pipeline');
my $contrib_extralibs = { 'libpq_pipeline' => ['ws2_32.lib'] };
my $contrib_extraincludes = {};
my $contrib_extrasource = {
'libpq_uri_regress' => ['src/interfaces/libpq/test/libpq_uri_regress.c'],
'libpq_testclient' => ['src/interfaces/libpq/test/libpq_testclient.c'],
+ 'libpq_pipeline' => ['src/interfaces/libpq/test/libpq_pipeline.c'],
};
my @contrib_excludes = (
'bool_plperl', 'commit_ts',
--
2.17.1
>From 38ee1523b794694bffeae573822d5793fbe80a3d Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sat, 2 Apr 2022 18:01:39 -0500
Subject: [PATCH 19/25] cirrus/linux: compile with -fsanitize
One concern about this is that the CompilerWarnings task currently
depends on it, and it seems important to avoid slowing down cfbot by
slowing down the Linux task so much that the Linux+Warnings takes longer
than windows. Maybe Warnings should depend on Freebsd, which is now the
fastest OS.
ci-os-only: linux
---
.cirrus.yml | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index f6763caecc4..d8f46a69296 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -211,8 +211,8 @@ task:
CC="ccache gcc" \
CXX="ccache g++" \
CLANG="ccache clang" \
- CFLAGS="-Og -ggdb" \
- CXXFLAGS="-Og -ggdb"
+ CFLAGS="-Og -ggdb -fsanitize=undefined,alignment -fno-sanitize-recover=all" \
+ CXXFLAGS="-Og -ggdb -fsanitize=undefined,alignment -fno-sanitize-recover=all"
EOF
build_script: |
@@ -484,7 +484,7 @@ task:
# To limit unnecessary work only run this once the normal linux test succeeds
depends_on:
- - Linux - Debian Bullseye
+ - FreeBSD - 13
env:
CPUS: 4
--
2.17.1
>From 8488c1be004ad2c43fbcfab3de11bee8e623451b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Mon, 17 Jan 2022 00:54:28 -0600
Subject: [PATCH 20/25] cirrus: code coverage
https://www.postgresql.org/message-id/202202111821.w3gqblvfp4pr%40alvherre.pgsql
https://www.postgresql.org/message-id/flat/20220409021853.gp24...@telsasoft.com
ci-os-only: linux
---
.cirrus.yml | 16 ++++++++++++++++
src/tools/ci/code-coverage-report | 29 +++++++++++++++++++++++++++++
2 files changed, 45 insertions(+)
create mode 100755 src/tools/ci/code-coverage-report
diff --git a/.cirrus.yml b/.cirrus.yml
index d8f46a69296..f66ce2d7146 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -28,6 +28,13 @@ env:
TEMP_CONFIG: ${CIRRUS_WORKING_DIR}/src/tools/ci/pg_ci_base.conf
PG_TEST_EXTRA: kerberos ldap ssl
+ # The commit that this branch is rebased on. There's no easy way to find this.
+ # This does the right thing for cfbot, which always squishes all patches into a single commit.
+ # And does the right thing for any 1-patch commits.
+ # Patch series manually submitted to cirrus would benefit from setting this to the
+ # number of patches in the series (or directly to the commit the series was rebased on).
+ BASE_COMMIT: HEAD~1
+
# What files to preserve in case tests fail
on_failure: &on_failure
@@ -185,6 +192,7 @@ task:
cat /proc/cmdline
ulimit -a -H && ulimit -a -S
export
+ git diff --name-only "$BASE_COMMIT"
create_user_script: |
useradd -m postgres
chown -R postgres:postgres .
@@ -204,6 +212,7 @@ task:
su postgres <<-EOF
./configure \
--enable-cassert --enable-debug --enable-tap-tests \
+ --enable-coverage \
--enable-nls \
\
${LINUX_CONFIGURE_FEATURES} \
@@ -227,6 +236,13 @@ task:
make -s ${CHECK} ${CHECKFLAGS} -j${TEST_JOBS}
EOF
+ # Build coverage report for files changed since the base commit.
+ generate_coverage_report_script: |
+ src/tools/ci/code-coverage-report "$BASE_COMMIT"
+
+ coverage_artifacts:
+ paths: ['coverage/**/*.html', 'coverage/**/*.png', 'coverage/**/*.gcov', 'coverage/**/*.css' ]
+
on_failure:
<<: *on_failure
cores_script: src/tools/ci/cores_backtrace.sh linux /tmp/cores
diff --git a/src/tools/ci/code-coverage-report b/src/tools/ci/code-coverage-report
new file mode 100755
index 00000000000..0dce149dcf8
--- /dev/null
+++ b/src/tools/ci/code-coverage-report
@@ -0,0 +1,29 @@
+#! /bin/sh
+# Called during the linux CI task to generate a code coverage report.
+set -e
+
+base_branch=$1
+changed=`git diff --name-only "$base_branch" '*.c'` ||
+ [ $? -eq 1 ]
+
+outdir=coverage
+mkdir "$outdir"
+
+# Coverage is shown only for changed files
+# This is useful to see coverage of newly-added code, but won't
+# show added/lost coverage in files which this patch doesn't modify.
+
+gcov=$outdir/coverage.gcov
+for f in $changed
+do
+ # Avoid removed files
+ [ -f "$f" ] || continue
+
+ lcov --quiet --capture --directory "$f"
+done >"$gcov"
+
+# Exit successfully if no relevant files were changed
+[ -s "$gcov" ] || exit 0
+
+genhtml "$gcov" --show-details --legend --quiet --num-spaces=4 --output-directory "$outdir" --title="Coverage report of files changed since: $base_branch"
+cp "$outdir"/index.html "$outdir"/00-index.html
--
2.17.1
>From 7e2a62cb662f80befa1ff3f4539786edb8def9c0 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sat, 26 Feb 2022 19:34:35 -0600
Subject: [PATCH 21/25] cirrus: build docs as a separate task..
This will run the doc build if any docs have changed, even if Linux
fails, to allow catch doc build failures.
This'll automatically show up as a separate "column" on cfbot.
Also, in the future, this will hopefully upload each patch's changed HTML docs
as an artifact, for easy review.
XX-os-only: html
---
.cirrus.yml | 49 +++++++++++++++++++++++++++++++++++--------------
1 file changed, 35 insertions(+), 14 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index f66ce2d7146..d4f3bd70785 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -614,20 +614,6 @@ task:
time make -s -j${BUILD_JOBS} world-bin
ccache --show-stats
- ###
- # Verify docs can be built
- ###
- # XXX: Only do this if there have been changes in doc/ since last build
- always:
- docs_build_script: |
- time ./configure \
- --cache gcc.cache \
- CC="ccache gcc" \
- CXX="ccache g++" \
- CLANG="ccache clang"
- make -s -j${BUILD_JOBS} clean
- time make -s -j${BUILD_JOBS} -C doc
-
###
# Verify headerscheck / cpluspluscheck succeed
#
@@ -653,3 +639,38 @@ task:
always:
upload_caches: ccache
+
+
+###
+# Verify docs can be built
+# changesInclude() will skip this task if none of the commits since
+# CIRRUS_LAST_GREEN_CHANGE touched any relevant files. The comparison appears
+# to be like "git log a..b -- ./file", not "git diff a..b -- ./file"
+###
+
+task:
+ name: Documentation
+
+ env:
+ CPUS: 1
+ BUILD_JOBS: 1
+
+ only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(docs|html).*'
+ skip: "!changesInclude('.cirrus.yml', 'doc/**')"
+
+ container:
+ image: $CONTAINER_REPO/linux_debian_bullseye_ci:latest
+ cpu: $CPUS
+ memory: 2G
+
+ sysinfo_script: |
+ id
+ uname -a
+ cat /proc/cmdline
+ ulimit -a -H && ulimit -a -S
+ export
+
+ # Exercise HTML and other docs:
+ docs_build_script: |
+ time ./configure
+ make -s -j${BUILD_JOBS} -C doc
--
2.17.1
>From fb53efe1b5a85b11438c425128b8df689150f33f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Sat, 26 Feb 2022 19:39:10 -0600
Subject: [PATCH 22/25] cirrus: upload changed html docs as artifacts
This could be done on the client side (cfbot). One advantage of doing it here
is that fewer docs are uploaded - many patches won't upload docs at all.
https://www.postgresql.org/message-id/flat/20220409021853.gp24...@telsasoft.com
https://cirrus-ci.com/task/5396696388599808
ci-os-only: html
---
.cirrus.yml | 19 +++++++++++++++++--
src/tools/ci/copy-changed-docs | 23 +++++++++++++++++++++++
2 files changed, 40 insertions(+), 2 deletions(-)
create mode 100755 src/tools/ci/copy-changed-docs
diff --git a/.cirrus.yml b/.cirrus.yml
index d4f3bd70785..e6e76ab8c9c 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -642,7 +642,7 @@ task:
###
-# Verify docs can be built
+# Verify docs can be built, and upload changed docs as artifacts
# changesInclude() will skip this task if none of the commits since
# CIRRUS_LAST_GREEN_CHANGE touched any relevant files. The comparison appears
# to be like "git log a..b -- ./file", not "git diff a..b -- ./file"
@@ -656,7 +656,7 @@ task:
BUILD_JOBS: 1
only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(docs|html).*'
- skip: "!changesInclude('.cirrus.yml', 'doc/**')"
+ #skip: "!changesInclude('.cirrus.yml', 'doc/**', 'src/tools/ci/copy-changed-docs')"
container:
image: $CONTAINER_REPO/linux_debian_bullseye_ci:latest
@@ -669,8 +669,23 @@ task:
cat /proc/cmdline
ulimit -a -H && ulimit -a -S
export
+ git diff --name-only "$BASE_COMMIT"
# Exercise HTML and other docs:
docs_build_script: |
time ./configure
make -s -j${BUILD_JOBS} -C doc
+ cp -r doc new-docs
+
+ # Build HTML docs from the base commit.
+ git checkout "$BASE_COMMIT" -- doc
+ #html-stamp
+ make -s -C doc clean
+ make -s -C doc html
+ cp -r doc old-docs
+
+ copy_changed_docs_script: |
+ src/tools/ci/copy-changed-docs "old-docs" "new-docs" "html_docs"
+
+ html_docs_artifacts:
+ paths: ['html_docs/*.html', 'html_docs/*.png', 'html_docs/*.css']
diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs
new file mode 100755
index 00000000000..1584f974d94
--- /dev/null
+++ b/src/tools/ci/copy-changed-docs
@@ -0,0 +1,23 @@
+#! /bin/sh
+# Copy HTML which differ between $old and $new into $outdir
+set -e
+
+old=$1
+new=$2
+outdir=$3
+
+mkdir "$outdir"
+cp "$new"/src/sgml/html/*.css "$new"/src/sgml/html/*.svg "$outdir"
+
+changed=`git diff --no-index --name-only "$old"/src/sgml/html "$new"/src/sgml/html` ||
+ [ $? -eq 1 ]
+
+for f in $changed
+do
+ # Avoid removed files
+ [ -f "$f" ] || continue
+
+ cp -v "$f" "$outdir"
+done
+
+exit 0
--
2.17.1
>From 48fc80428f10fa8b4feeb733baaf662b105d2b4f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Mon, 28 Feb 2022 23:18:19 -0600
Subject: [PATCH 23/25] f!html: index file
This allows linking to the artifacts from the last successful build, which
itself allows *not* rebuilding when sources haven't changed.
ci-os-only: html,
xreebsd
---
.cirrus.yml | 3 ++-
src/tools/ci/copy-changed-docs | 45 ++++++++++++++++++++++++++++++++--
2 files changed, 45 insertions(+), 3 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index e6e76ab8c9c..0421d56ca70 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -656,7 +656,8 @@ task:
BUILD_JOBS: 1
only_if: $CIRRUS_CHANGE_MESSAGE !=~ '.*\nci-os-only:.*' || $CIRRUS_CHANGE_MESSAGE =~ '.*\nci-os-only:[^\n]*(docs|html).*'
- #skip: "!changesInclude('.cirrus.yml', 'doc/**', 'src/tools/ci/copy-changed-docs')"
+ skip: "!changesInclude('.cirrus.yml', 'doc/**', 'src/tools/ci/copy-changed-docs')"
+ #skip: "!changesInclude('doc/**')"
container:
image: $CONTAINER_REPO/linux_debian_bullseye_ci:latest
diff --git a/src/tools/ci/copy-changed-docs b/src/tools/ci/copy-changed-docs
index 1584f974d94..0efad386cca 100755
--- a/src/tools/ci/copy-changed-docs
+++ b/src/tools/ci/copy-changed-docs
@@ -5,10 +5,27 @@ set -e
old=$1
new=$2
outdir=$3
+branch=$CIRRUS_BRANCH
+
+# The index is large and changes often
+skippages="bookindex.html"
mkdir "$outdir"
cp "$new"/src/sgml/html/*.css "$new"/src/sgml/html/*.svg "$outdir"
+# The index is useful to allow a static link (not specific to a cirrus run) to the artifacts for the most-recent, successful CI run for a branch
+branchurl=https://api.cirrus-ci.com/v1/artifact/github/$CIRRUS_REPO_FULL_NAME/$CIRRUS_TASK_NAME/html_docs/html_docs/00-index.html?branch=$branch
+
+index="$outdir/00-index.html"
+cat >"$index" <<EOF
+<html>
+<head><title>Index of docs changed since: $BASE_COMMIT</title></head>
+<body>
+<!-- A link to documentation for the most recent successful build: $branchurl -->
+<h1>Index of docs changed since: $BASE_COMMIT</h1>
+<ul>
+EOF
+
changed=`git diff --no-index --name-only "$old"/src/sgml/html "$new"/src/sgml/html` ||
[ $? -eq 1 ]
@@ -17,7 +34,31 @@ do
# Avoid removed files
[ -f "$f" ] || continue
- cp -v "$f" "$outdir"
-done
+ echo "$f" |grep -Ew "$skippages" >/dev/null &&
+ continue
+
+ cp -v "$f" "$outdir" >&2
+ fn=${f##*/}
+ # ?branch=... is needed when accessing the artifacts for the static link for the branch
+ # It's not needed and ignored if accessing artifacts for *this* CI run
+ echo "<li><a href='$fn?branch=$branch'>$fn</a>"
+done >>"$index"
+
+github=https://github.com/$CIRRUS_REPO_FULL_NAME/commit
+cirrus=https://cirrus-ci.com/build
+
+cat >>"$index" <<EOF
+</ul>
+<hr>
+<code>
+<br>This file was written on: `date --rfc-822 --utc`
+<br>CIRRUS_CHANGE_TITLE: $CIRRUS_CHANGE_TITLE
+<br>CIRRUS_CHANGE_IN_REPO: <a href="$github/$CIRRUS_CHANGE_IN_REPO">$CIRRUS_CHANGE_IN_REPO</a>
+<br>CIRRUS_BUILD_ID: <a href="$cirrus/$CIRRUS_BUILD_ID">$CIRRUS_BUILD_ID</a>
+<br>CIRRUS_LAST_GREEN_CHANGE: <a href="$github/$CIRRUS_LAST_GREEN_CHANGE">$CIRRUS_LAST_GREEN_CHANGE</a>
+<br>CIRRUS_LAST_GREEN_BUILD_ID: <a href="$cirrus/$CIRRUS_LAST_GREEN_BUILD_ID">$CIRRUS_LAST_GREEN_BUILD_ID</a>
+</code>
+</body></html>
+EOF
exit 0
--
2.17.1
>From 9dc4bac25adce324ccb34b421570a6a76ceef42d Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Tue, 19 Jul 2022 12:38:45 -0500
Subject: [PATCH 24/25] cirrus/warnings: use ./configure cache in
headerscheck..
This is desirable since since configure is slow. It's necessary for the
environment variables to match, so disable ccache a different way.
ci-os-only: warnings
---
.cirrus.yml | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 0421d56ca70..81822c58bec 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -627,11 +627,13 @@ task:
###
always:
headers_headerscheck_script: |
+ export CCACHE_DISABLE=1
time ./configure \
+ --cache gcc.cache \
${LINUX_CONFIGURE_FEATURES} \
--without-icu \
--quiet \
- CC="gcc" CXX"=g++" CLANG="clang"
+ CC="ccache gcc" CXX="ccache g++" CLANG="ccache clang"
make -s -j${BUILD_JOBS} clean
time make -s headerscheck EXTRAFLAGS='-fmax-errors=10'
headers_cpluspluscheck_script: |
--
2.17.1
>From 11b56a11ce884bfb2adc03185b9d0b48b8fec2f2 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryz...@telsasoft.com>
Date: Tue, 19 Jul 2022 12:38:45 -0500
Subject: [PATCH 25/25] cirrus/warnings: move use a single/common 'always'
block
ci-os-only: warnings
---
.cirrus.yml | 48 +++++++++++++++++++++---------------------------
1 file changed, 21 insertions(+), 27 deletions(-)
diff --git a/.cirrus.yml b/.cirrus.yml
index 81822c58bec..25251db688b 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -530,11 +530,6 @@ task:
clang -v
export
- ccache_cache:
- folder: $CCACHE_DIR
- fingerprint_key: ccache/warnings
- reupload_on_changes: true
-
setup_additional_packages_script: |
#apt-get update
#DEBIAN_FRONTEND=noninteractive apt-get -y install ...
@@ -549,8 +544,13 @@ task:
# different compilers to build with different combinations of dtrace on/off
# and cassert on/off.
- # gcc, cassert off, dtrace on
always:
+ ccache_cache:
+ folder: $CCACHE_DIR
+ fingerprint_key: ccache/warnings
+ reupload_on_changes: true
+
+ # gcc, cassert off, dtrace on
gcc_warning_script: |
time ./configure \
--cache gcc.cache \
@@ -562,8 +562,7 @@ task:
time make -s -j${BUILD_JOBS} world-bin
ccache --show-stats
- # gcc, cassert on, dtrace off
- always:
+ # gcc, cassert on, dtrace off
gcc_a_warning_script: |
time ./configure \
--cache gcc.cache \
@@ -575,8 +574,7 @@ task:
time make -s -j${BUILD_JOBS} world-bin
ccache --show-stats
- # clang, cassert off, dtrace off
- always:
+ # clang, cassert off, dtrace off
clang_warning_script: |
time ./configure \
--cache clang.cache \
@@ -587,8 +585,7 @@ task:
time make -s -j${BUILD_JOBS} world-bin
ccache --show-stats
- # clang, cassert on, dtrace on
- always:
+ # clang, cassert on, dtrace on
clang_a_warning_script: |
time ./configure \
--cache clang.cache \
@@ -601,8 +598,7 @@ task:
time make -s -j${BUILD_JOBS} world-bin
ccache --show-stats
- # cross-compile to windows
- always:
+ # cross-compile to windows
mingw_cross_warning_script: |
time ./configure \
--host=x86_64-w64-mingw32 \
@@ -614,18 +610,17 @@ task:
time make -s -j${BUILD_JOBS} world-bin
ccache --show-stats
- ###
- # Verify headerscheck / cpluspluscheck succeed
- #
- # - Don't use ccache, the files are uncacheable, polluting ccache's
- # cache
- # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose
- # - XXX have to disable ICU to avoid errors:
- # https://postgr.es/m/20220323002024.f2g6tivduzrktgfa%40alap3.anarazel.de
- # - XXX: the -Wno-register avoids verbose warnings:
- # https://postgr.es/m/20220308181837.aun3tdtdvao4vb7o%40alap3.anarazel.de
- ###
- always:
+ ###
+ # Verify headerscheck / cpluspluscheck succeed
+ #
+ # - Don't use ccache, the files are uncacheable, polluting ccache's
+ # cache
+ # - Use -fmax-errors, as particularly cpluspluscheck can be very verbose
+ # - XXX have to disable ICU to avoid errors:
+ # https://postgr.es/m/20220323002024.f2g6tivduzrktgfa%40alap3.anarazel.de
+ # - XXX: the -Wno-register avoids verbose warnings:
+ # https://postgr.es/m/20220308181837.aun3tdtdvao4vb7o%40alap3.anarazel.de
+ ###
headers_headerscheck_script: |
export CCACHE_DISABLE=1
time ./configure \
@@ -639,7 +634,6 @@ task:
headers_cpluspluscheck_script: |
time make -s cpluspluscheck EXTRAFLAGS='-Wno-register -fmax-errors=10'
- always:
upload_caches: ccache
--
2.17.1
