On Fri, Dec 10, 2021 at 10:36 AM Andres Freund <and...@anarazel.de> wrote:
> Seems like we ought to add a tiny tap test or such for this - otherwise we
> won't have any coverage of "normal" tablespaces? I don't think they need to be
> exercised as part of the normal tests, but having some very basic testing
> in a tap test seems worthwhile.
Good idea, that was bothering me too. Done.
> > - print $conf "max_connections = 10\n";
> > + print $conf "max_connections = 25\n";
> What's the relation of this to the rest?
Someone decided that allow_streaming should imply max_connections =
10, but we need ~20 to run the parallel regression test schedule.
However, I can just as easily move that to a local adjustment in the
TAP test file. Done, like so:
+$node_primary->adjust_conf('postgresql.conf', 'max_connections', '25', 1);
> Absurd nitpick: What's the deal with using "" for one part, and '' for the
> rest?
Fixed.
> Separately: I think the case of seeing diffs will be too hard to debug like
> this, as the difference isn't shown afaict?
Seems to be OK. The output goes to
src/test/recovery/tmp_check/log/regress_log_027_stream_regress, so for
example if you comment out the bit that deals with SEQUENCE caching
you'll see:
# Running: pg_dump -f
/usr/home/tmunro/projects/postgresql/src/test/recovery/primary.dump
--no-sync -p 63693 regression
ok 2 - dump primary server
# Running: pg_dump -f
/usr/home/tmunro/projects/postgresql/src/test/recovery/standby.dump
--no-sync -p 63694 regression
ok 3 - dump standby server
# Running: diff
/usr/home/tmunro/projects/postgresql/src/test/recovery/primary.dump
/usr/home/tmunro/projects/postgresql/src/test/recovery/standby.dump
436953c436953
< SELECT pg_catalog.setval('public.clstr_tst_s_rf_a_seq', 32, true);
---
> SELECT pg_catalog.setval('public.clstr_tst_s_rf_a_seq', 33, true);
... more hunks ...
And from the previous email:
On Fri, Dec 10, 2021 at 10:35 AM Thomas Munro <thomas.mu...@gmail.com> wrote:
> On Fri, Dec 10, 2021 at 8:38 AM Andres Freund <and...@anarazel.de> wrote:
> > Personally I'd rather put relative tablespaces into a dedicated directory or
> > just into pg_tblspc, but without a symlink. Some tools need to understand
> > tablespace layout etc, and having them in a directory that, by the name,
> > will
> > also contain other things seems likely to cause confusion.
Ok, in this version I have a developer-only GUC
allow_in_place_tablespaces instead. If you turn it on, you can do:
CREATE TABLESPACE regress_blah LOCATION = '';
... and then pg_tblspc/OID is created directly as a directory. Not
allowed by default or documented.
From 345fe049309ed84a377cda8e5a3c6df10482ae9a Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Fri, 10 Dec 2021 11:45:24 +1300
Subject: [PATCH v9 1/6] Allow "in place" tablespaces.
Provide a developer-only GUC allow_in_place_tablespaces, disabled by
default. When enabled, tablespaces can be created with an empty
LOCATION string, meaning that they should be created as a directory
directly beneath pg_tblspc. This can be used for new testing scenarios,
in a follow-up patch. Not intended for end-user usage, since it might
confuse backup tools that expect symlinks.
Discussion: https://postgr.es/m/CA%2BhUKGKpRWQ9SxdxxDmTBCJoR0YnFpMBe7kyzY8SUQk%2BHeskxg%40mail.gmail.com
---
src/backend/commands/tablespace.c | 36 +++++++++++++++++++++++++------
src/backend/utils/misc/guc.c | 12 +++++++++++
src/include/commands/tablespace.h | 2 ++
3 files changed, 44 insertions(+), 6 deletions(-)
diff --git a/src/backend/commands/tablespace.c b/src/backend/commands/tablespace.c
index 4b96eec9df..7a950c75a7 100644
--- a/src/backend/commands/tablespace.c
+++ b/src/backend/commands/tablespace.c
@@ -87,6 +87,7 @@
/* GUC variables */
char *default_tablespace = NULL;
char *temp_tablespaces = NULL;
+bool allow_in_place_tablespaces = false;
static void create_tablespace_directories(const char *location,
@@ -241,6 +242,7 @@ CreateTableSpace(CreateTableSpaceStmt *stmt)
char *location;
Oid ownerId;
Datum newOptions;
+ bool in_place;
/* Must be superuser */
if (!superuser())
@@ -266,12 +268,15 @@ CreateTableSpace(CreateTableSpaceStmt *stmt)
(errcode(ERRCODE_INVALID_NAME),
errmsg("tablespace location cannot contain single quotes")));
+ in_place = allow_in_place_tablespaces && strlen(location) == 0;
+
/*
* Allowing relative paths seems risky
*
- * this also helps us ensure that location is not empty or whitespace
+ * This also helps us ensure that location is not empty or whitespace,
+ * unless specifying a developer-only in-place tablespace.
*/
- if (!is_absolute_path(location))
+ if (!in_place && !is_absolute_path(location))
ereport(ERROR,
(errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
errmsg("tablespace location must be an absolute path")));
@@ -590,16 +595,35 @@ create_tablespace_directories(const char *location, const Oid tablespaceoid)
char *linkloc;
char *location_with_version_dir;
struct stat st;
+ bool in_place;
linkloc = psprintf("pg_tblspc/%u", tablespaceoid);
- location_with_version_dir = psprintf("%s/%s", location,
+
+ /*
+ * If we're asked to make an 'in place' tablespace, create the directory
+ * directly where the symlink would normally go. This is a developer-only
+ * option for now, to facilitate regression testing.
+ */
+ in_place =
+ (allow_in_place_tablespaces || InRecovery) && strlen(location) == 0;
+
+ if (in_place)
+ {
+ if (MakePGDirectory(linkloc) < 0 && errno != EEXIST)
+ ereport(ERROR,
+ (errcode_for_file_access(),
+ errmsg("could not create directory \"%s\": %m",
+ linkloc)));
+ }
+
+ location_with_version_dir = psprintf("%s/%s", in_place ? linkloc : location,
TABLESPACE_VERSION_DIRECTORY);
/*
* Attempt to coerce target directory to safe permissions. If this fails,
* it doesn't exist or has the wrong owner.
*/
- if (chmod(location, pg_dir_create_mode) != 0)
+ if (!in_place && chmod(location, pg_dir_create_mode) != 0)
{
if (errno == ENOENT)
ereport(ERROR,
@@ -648,13 +672,13 @@ create_tablespace_directories(const char *location, const Oid tablespaceoid)
/*
* In recovery, remove old symlink, in case it points to the wrong place.
*/
- if (InRecovery)
+ if (!in_place && InRecovery)
remove_tablespace_symlink(linkloc);
/*
* Create the symlink under PGDATA
*/
- if (symlink(location, linkloc) < 0)
+ if (!in_place && symlink(location, linkloc) < 0)
ereport(ERROR,
(errcode_for_file_access(),
errmsg("could not create symbolic link \"%s\": %m",
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index ee6a838b3a..0fafacd726 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -46,6 +46,7 @@
#include "catalog/storage.h"
#include "commands/async.h"
#include "commands/prepare.h"
+#include "commands/tablespace.h"
#include "commands/trigger.h"
#include "commands/user.h"
#include "commands/vacuum.h"
@@ -1963,6 +1964,17 @@ static struct config_bool ConfigureNamesBool[] =
NULL, NULL, NULL
},
+ {
+ {"allow_in_place_tablespaces", PGC_SUSET, DEVELOPER_OPTIONS,
+ gettext_noop("Allows tablespaces directly inside pg_tblspc, for testing."),
+ NULL,
+ GUC_NOT_IN_SAMPLE
+ },
+ &allow_in_place_tablespaces,
+ false,
+ NULL, NULL, NULL
+ },
+
{
{"lo_compat_privileges", PGC_SUSET, COMPAT_OPTIONS_PREVIOUS,
gettext_noop("Enables backward compatibility mode for privilege checks on large objects."),
diff --git a/src/include/commands/tablespace.h b/src/include/commands/tablespace.h
index 49cfc8479a..f33ba51bd6 100644
--- a/src/include/commands/tablespace.h
+++ b/src/include/commands/tablespace.h
@@ -19,6 +19,8 @@
#include "lib/stringinfo.h"
#include "nodes/parsenodes.h"
+extern bool allow_in_place_tablespaces;
+
/* XLOG stuff */
#define XLOG_TBLSPC_CREATE 0x00
#define XLOG_TBLSPC_DROP 0x10
--
2.33.1
From 02554d87073ca3019558b5a02bd4b89bcb1e7e73 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Wed, 4 Aug 2021 22:17:54 +1200
Subject: [PATCH v9 2/6] Use in-place tablespaces in regression test.
Remove the machinery from pg_regress that manages the testtablespace
directory. Instead, use "in-place" tablespaces, because they work
correctly when there is a streaming replica running on the same host.
Discussion: https://postgr.es/m/CA%2BhUKGKpRWQ9SxdxxDmTBCJoR0YnFpMBe7kyzY8SUQk%2BHeskxg%40mail.gmail.com
---
src/test/regress/GNUmakefile | 3 +-
.../tablespace.out} | 17 ++++++--
src/test/regress/pg_regress.c | 40 -------------------
.../tablespace.source => sql/tablespace.sql} | 18 +++++++--
src/tools/msvc/vcregress.pl | 2 -
5 files changed, 30 insertions(+), 50 deletions(-)
rename src/test/regress/{output/tablespace.source => expected/tablespace.out} (97%)
rename src/test/regress/{input/tablespace.source => sql/tablespace.sql} (96%)
diff --git a/src/test/regress/GNUmakefile b/src/test/regress/GNUmakefile
index fe6e0c98aa..d014b4a35d 100644
--- a/src/test/regress/GNUmakefile
+++ b/src/test/regress/GNUmakefile
@@ -119,7 +119,7 @@ submake-contrib-spi: | submake-libpgport submake-generated-headers
## Run tests
##
-REGRESS_OPTS = --dlpath=. --max-concurrent-tests=20 --make-testtablespace-dir \
+REGRESS_OPTS = --dlpath=. --max-concurrent-tests=20 \
$(EXTRA_REGRESS_OPTS)
check: all
@@ -163,5 +163,4 @@ clean distclean maintainer-clean: clean-lib
rm -f pg_regress_main.o pg_regress.o pg_regress$(X)
# things created by various check targets
rm -f $(output_files) $(input_files)
- rm -rf testtablespace
rm -rf $(pg_regress_clean_files)
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/expected/tablespace.out
similarity index 97%
rename from src/test/regress/output/tablespace.source
rename to src/test/regress/expected/tablespace.out
index e7629d470e..4169c36cde 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/expected/tablespace.out
@@ -1,7 +1,18 @@
+-- relative tablespace locations are not allowed
+CREATE TABLESPACE regress_tblspace LOCATION 'relative'; -- fail
+ERROR: tablespace location must be an absolute path
+-- empty tablespace locations are not usually allowed
+CREATE TABLESPACE regress_tblspace LOCATION ''; -- fail
+ERROR: tablespace location must be an absolute path
+-- as a special developer-only option to allow us to use tablespaces
+-- with streaming replication on the same server, an empty location
+-- can be allowed as a way to say that the tablespace should be created
+-- as a directory in pg_tblspc, rather than being a symlink
+SET allow_in_place_tablespaces = true;
-- create a tablespace using WITH clause
-CREATE TABLESPACE regress_tblspacewith LOCATION '@testtablespace@' WITH (some_nonexistent_parameter = true); -- fail
+CREATE TABLESPACE regress_tblspacewith LOCATION '' WITH (some_nonexistent_parameter = true); -- fail
ERROR: unrecognized parameter "some_nonexistent_parameter"
-CREATE TABLESPACE regress_tblspacewith LOCATION '@testtablespace@' WITH (random_page_cost = 3.0); -- ok
+CREATE TABLESPACE regress_tblspacewith LOCATION '' WITH (random_page_cost = 3.0); -- ok
-- check to see the parameter was used
SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';
spcoptions
@@ -12,7 +23,7 @@ SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';
-- drop the tablespace so we can re-use the location
DROP TABLESPACE regress_tblspacewith;
-- create a tablespace we can use
-CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
+CREATE TABLESPACE regress_tblspace LOCATION '';
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
ALTER TABLESPACE regress_tblspace SET (some_nonexistent_parameter = true); -- fail
diff --git a/src/test/regress/pg_regress.c b/src/test/regress/pg_regress.c
index 2c8a600bad..7c1404dac1 100644
--- a/src/test/regress/pg_regress.c
+++ b/src/test/regress/pg_regress.c
@@ -477,7 +477,6 @@ replace_string(StringInfo string, const char *replace, const char *replacement)
static void
convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const char *dest_subdir, const char *suffix)
{
- char testtablespace[MAXPGPATH];
char indir[MAXPGPATH];
char outdir_sub[MAXPGPATH];
char **name;
@@ -506,9 +505,6 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
if (!directory_exists(outdir_sub))
make_directory(outdir_sub);
- /* We might need to replace @testtablespace@ */
- snprintf(testtablespace, MAXPGPATH, "%s/testtablespace", outputdir);
-
/* finally loop on each file and do the replacement */
for (name = names; *name; name++)
{
@@ -554,7 +550,6 @@ convert_sourcefiles_in(const char *source_subdir, const char *dest_dir, const ch
{
replace_string(&line, "@abs_srcdir@", inputdir);
replace_string(&line, "@abs_builddir@", outputdir);
- replace_string(&line, "@testtablespace@", testtablespace);
replace_string(&line, "@libdir@", dlpath);
replace_string(&line, "@DLSUFFIX@", DLSUFFIX);
fputs(line.data, outfile);
@@ -587,32 +582,6 @@ convert_sourcefiles(void)
convert_sourcefiles_in("output", outputdir, "expected", "out");
}
-/*
- * Clean out the test tablespace dir, or create it if it doesn't exist.
- *
- * On Windows, doing this cleanup here makes it possible to run the
- * regression tests under a Windows administrative user account with the
- * restricted token obtained when starting pg_regress.
- */
-static void
-prepare_testtablespace_dir(void)
-{
- char testtablespace[MAXPGPATH];
-
- snprintf(testtablespace, MAXPGPATH, "%s/testtablespace", outputdir);
-
- if (directory_exists(testtablespace))
- {
- if (!rmtree(testtablespace, true))
- {
- fprintf(stderr, _("\n%s: could not remove test tablespace \"%s\"\n"),
- progname, testtablespace);
- exit(2);
- }
- }
- make_directory(testtablespace);
-}
-
/*
* Scan resultmap file to find which platform-specific expected files to use.
*
@@ -2152,7 +2121,6 @@ help(void)
printf(_(" --launcher=CMD use CMD as launcher of psql\n"));
printf(_(" --load-extension=EXT load the named extension before running the\n"));
printf(_(" tests; can appear multiple times\n"));
- printf(_(" --make-testtablespace-dir create testtablespace directory\n"));
printf(_(" --max-connections=N maximum number of concurrent connections\n"));
printf(_(" (default is 0, meaning unlimited)\n"));
printf(_(" --max-concurrent-tests=N maximum number of concurrent tests in schedule\n"));
@@ -2211,12 +2179,10 @@ regression_main(int argc, char *argv[],
{"load-extension", required_argument, NULL, 22},
{"config-auth", required_argument, NULL, 24},
{"max-concurrent-tests", required_argument, NULL, 25},
- {"make-testtablespace-dir", no_argument, NULL, 26},
{NULL, 0, NULL, 0}
};
bool use_unix_sockets;
- bool make_testtablespace_dir = false;
_stringlist *sl;
int c;
int i;
@@ -2342,9 +2308,6 @@ regression_main(int argc, char *argv[],
case 25:
max_concurrent_tests = atoi(optarg);
break;
- case 26:
- make_testtablespace_dir = true;
- break;
default:
/* getopt_long already emitted a complaint */
fprintf(stderr, _("\nTry \"%s -h\" for more information.\n"),
@@ -2397,9 +2360,6 @@ regression_main(int argc, char *argv[],
unlimit_core_size();
#endif
- if (make_testtablespace_dir)
- prepare_testtablespace_dir();
-
if (temp_instance)
{
FILE *pg_conf;
diff --git a/src/test/regress/input/tablespace.source b/src/test/regress/sql/tablespace.sql
similarity index 96%
rename from src/test/regress/input/tablespace.source
rename to src/test/regress/sql/tablespace.sql
index cb9774ecc8..a42e33ed72 100644
--- a/src/test/regress/input/tablespace.source
+++ b/src/test/regress/sql/tablespace.sql
@@ -1,6 +1,18 @@
+-- relative tablespace locations are not allowed
+CREATE TABLESPACE regress_tblspace LOCATION 'relative'; -- fail
+
+-- empty tablespace locations are not usually allowed
+CREATE TABLESPACE regress_tblspace LOCATION ''; -- fail
+
+-- as a special developer-only option to allow us to use tablespaces
+-- with streaming replication on the same server, an empty location
+-- can be allowed as a way to say that the tablespace should be created
+-- as a directory in pg_tblspc, rather than being a symlink
+SET allow_in_place_tablespaces = true;
+
-- create a tablespace using WITH clause
-CREATE TABLESPACE regress_tblspacewith LOCATION '@testtablespace@' WITH (some_nonexistent_parameter = true); -- fail
-CREATE TABLESPACE regress_tblspacewith LOCATION '@testtablespace@' WITH (random_page_cost = 3.0); -- ok
+CREATE TABLESPACE regress_tblspacewith LOCATION '' WITH (some_nonexistent_parameter = true); -- fail
+CREATE TABLESPACE regress_tblspacewith LOCATION '' WITH (random_page_cost = 3.0); -- ok
-- check to see the parameter was used
SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';
@@ -9,7 +21,7 @@ SELECT spcoptions FROM pg_tablespace WHERE spcname = 'regress_tblspacewith';
DROP TABLESPACE regress_tblspacewith;
-- create a tablespace we can use
-CREATE TABLESPACE regress_tblspace LOCATION '@testtablespace@';
+CREATE TABLESPACE regress_tblspace LOCATION '';
-- try setting and resetting some properties for the new tablespace
ALTER TABLESPACE regress_tblspace SET (random_page_cost = 1.0, seq_page_cost = 1.1);
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index 5975e7c9cd..bf66083f73 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -133,7 +133,6 @@ sub installcheck_internal
"--bindir=../../../$Config/psql",
"--schedule=${schedule}_schedule",
"--max-concurrent-tests=20",
- "--make-testtablespace-dir",
"--encoding=SQL_ASCII",
"--no-locale");
push(@args, $maxconn) if $maxconn;
@@ -168,7 +167,6 @@ sub check
"--bindir=",
"--schedule=${schedule}_schedule",
"--max-concurrent-tests=20",
- "--make-testtablespace-dir",
"--encoding=SQL_ASCII",
"--no-locale",
"--temp-instance=./tmp_check");
--
2.33.1
From 0f8e296db9c870f9c59d62dad4cb07f8b7137d05 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Fri, 10 Dec 2021 12:22:55 +1300
Subject: [PATCH v9 3/6] Add new simple TAP test for tablespaces.
The tablespace tests in the main regression tests have been changed to
use "in-place" tablespaces, so that they work when streamed to a replica
on the same host. Add a new TAP test that exercises tablespaces with
absolute paths (but couldn't be replicated), for coverage.
Discussion: https://postgr.es/m/CA%2BhUKGKpRWQ9SxdxxDmTBCJoR0YnFpMBe7kyzY8SUQk%2BHeskxg%40mail.gmail.com
---
.../modules/test_misc/t/002_tablespace.pl | 49 +++++++++++++++++++
1 file changed, 49 insertions(+)
create mode 100644 src/test/modules/test_misc/t/002_tablespace.pl
diff --git a/src/test/modules/test_misc/t/002_tablespace.pl b/src/test/modules/test_misc/t/002_tablespace.pl
new file mode 100644
index 0000000000..3864e5c24b
--- /dev/null
+++ b/src/test/modules/test_misc/t/002_tablespace.pl
@@ -0,0 +1,49 @@
+# Simple tablespace tests that can't be replicated on the same host
+# due to the use of absolute paths, so we keep them out of the regular
+# regression tests.
+
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 6;
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+$node->init;
+$node->start;
+
+# Create a directory to hold the tablespace.
+my $LOCATION = $node->basedir() . "/testtablespace";
+mkdir($LOCATION);
+
+my $result;
+
+# Create a tablespace with an absolute path
+$result = $node->psql('postgres',
+ "CREATE TABLESPACE regress_tablespace LOCATION '$LOCATION'");
+ok($result == 0, 'create tablespace with absolute path');
+
+# Can't create a tablespace where there is one already
+$result = $node->psql('postgres',
+ "CREATE TABLESPACE regress_tablespace LOCATION '$LOCATION'");
+ok($result != 0, 'clobber tablespace with absolute path');
+
+# Create table in it
+$result = $node->psql('postgres',
+ "CREATE TABLE t () TABLESPACE regress_tablespace");
+ok($result == 0, 'create table in tablespace with absolute path');
+
+# Can't drop a tablespace that still has a table in it
+$result = $node->psql('postgres',
+ "DROP TABLESPACE regress_tablespace LOCATION '$LOCATION'");
+ok($result != 0, 'drop tablespace with absolute path');
+
+# Drop the table
+$result = $node->psql('postgres', "DROP TABLE t");
+ok($result == 0, 'drop table in tablespace with absolute path');
+
+# Drop the tablespace
+$result = $node->psql('postgres', "DROP TABLESPACE regress_tablespace");
+ok($result == 0, 'drop tablespace with absolute path');
+
+$node->stop;
--
2.33.1
From 48d28b222c518330cf6cea262f2b3ab4ee504238 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Tue, 5 Oct 2021 21:01:02 +1300
Subject: [PATCH v9 4/6] Test replay of regression tests.
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Add a new TAP test under src/test/recovery to run the standard
regression tests while a streaming replica replays the WAL. This
provides a basic workout for WAL decoding and redo code, and compares
the replicated result.
Optionally, enable (expensive) wal_consistency_checking if listed in
the env variable PG_TEST_EXTRA.
Reviewed-by: 綱川 貴之 (Takayuki Tsunakawa) <tsunakawa.ta...@fujitsu.com>
Reviewed-by: Andres Freund <and...@anarazel.de>
Reviewed-by: Andrew Dunstan <and...@dunslane.net>
Reviewed-by: Tom Lane <t...@sss.pgh.pa.us>
Reviewed-by: Anastasia Lubennikova <lubennikov...@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGKpRWQ9SxdxxDmTBCJoR0YnFpMBe7kyzY8SUQk%2BHeskxg%40mail.gmail.com
---
doc/src/sgml/regress.sgml | 11 ++++
src/test/recovery/Makefile | 6 +-
src/test/recovery/t/027_stream_regress.pl | 76 +++++++++++++++++++++++
src/tools/msvc/vcregress.pl | 2 +
4 files changed, 94 insertions(+), 1 deletion(-)
create mode 100644 src/test/recovery/t/027_stream_regress.pl
diff --git a/doc/src/sgml/regress.sgml b/doc/src/sgml/regress.sgml
index 724ef566e7..64fc81776d 100644
--- a/doc/src/sgml/regress.sgml
+++ b/doc/src/sgml/regress.sgml
@@ -289,6 +289,17 @@ make check-world PG_TEST_EXTRA='kerberos ldap ssl'
</para>
</listitem>
</varlistentry>
+
+ <varlistentry>
+ <term><literal>wal_consistency_checking</literal></term>
+ <listitem>
+ <para>
+ Uses <literal>wal_consistency_checking=all</literal> while running
+ some of the tests under <filename>src/test/recovery</filename>. Not
+ enabled by default because it is resource intensive.
+ </para>
+ </listitem>
+ </varlistentry>
</variablelist>
Tests for features that are not supported by the current build
diff --git a/src/test/recovery/Makefile b/src/test/recovery/Makefile
index 288c04b861..c557018b38 100644
--- a/src/test/recovery/Makefile
+++ b/src/test/recovery/Makefile
@@ -15,10 +15,14 @@ subdir = src/test/recovery
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-# required for 017_shm.pl
+# required for 017_shm.pl and 027_stream_regress.pl
REGRESS_SHLIB=$(abs_top_builddir)/src/test/regress/regress$(DLSUFFIX)
export REGRESS_SHLIB
+# required for 027_stream_regress.pl
+REGRESS_OUTPUTDIR=$(abs_top_builddir)/src/test/recovery
+export REGRESS_OUTPUTDIR
+
check:
$(prove_check)
diff --git a/src/test/recovery/t/027_stream_regress.pl b/src/test/recovery/t/027_stream_regress.pl
new file mode 100644
index 0000000000..74993cd30e
--- /dev/null
+++ b/src/test/recovery/t/027_stream_regress.pl
@@ -0,0 +1,76 @@
+# Run the standard regression tests with streaming replication
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More tests => 4;
+use File::Basename;
+
+# Initialize primary node
+my $node_primary = PostgreSQL::Test::Cluster->new('primary');
+$node_primary->init(allows_streaming => 1);
+$node_primary->adjust_conf('postgresql.conf', 'max_connections', '25', 1);
+
+# WAL consistency checking is resource intensive so require opt-in with the
+# PG_TEST_EXTRA environment variable.
+if ($ENV{PG_TEST_EXTRA} &&
+ $ENV{PG_TEST_EXTRA} =~ m/\bwal_consistency_checking\b/) {
+ $node_primary->append_conf('postgresql.conf',
+ 'wal_consistency_checking = all');
+}
+
+$node_primary->start;
+is( $node_primary->psql(
+ 'postgres',
+ qq[SELECT pg_create_physical_replication_slot('standby_1');]),
+ 0,
+ 'physical slot created on primary');
+my $backup_name = 'my_backup';
+
+# Take backup
+$node_primary->backup($backup_name);
+
+# Create streaming standby linking to primary
+my $node_standby_1 = PostgreSQL::Test::Cluster->new('standby_1');
+$node_standby_1->init_from_backup($node_primary, $backup_name,
+ has_streaming => 1);
+$node_standby_1->append_conf('postgresql.conf',
+ "primary_slot_name = standby_1");
+$node_standby_1->start;
+
+my $dlpath = PostgreSQL::Test::Utils::perl2host(dirname($ENV{REGRESS_SHLIB}));
+my $outputdir = PostgreSQL::Test::Utils::perl2host($ENV{REGRESS_OUTPUTDIR});
+
+# Run the regression tests against the primary.
+system_or_bail($ENV{PG_REGRESS},
+ "--dlpath=" . $dlpath,
+ "--bindir=",
+ "--port=" . $node_primary->port,
+ "--schedule=../regress/parallel_schedule",
+ "--inputdir=../regress",
+ "--outputdir=" . $outputdir);
+
+# Clobber all sequences with their next value, so that we don't have
+# differences between nodes due to caching.
+$node_primary->psql('regression',
+ "select setval(seqrelid, nextval(seqrelid)) from pg_sequence");
+
+# Wait for standby to catch up
+$node_primary->wait_for_catchup($node_standby_1, 'replay',
+ $node_primary->lsn('insert'));
+
+# Perform a logical dump of primary and standby, and check that they match
+command_ok(
+ [ 'pg_dump', '-f', $outputdir . '/primary.dump', '--no-sync',
+ '-p', $node_primary->port, 'regression' ],
+ 'dump primary server');
+command_ok(
+ [ 'pg_dump', '-f', $outputdir . '/standby.dump', '--no-sync',
+ '-p', $node_standby_1->port, 'regression' ],
+ 'dump standby server');
+command_ok(
+ [ 'diff', $outputdir . '/primary.dump', $outputdir . '/standby.dump' ],
+ 'compare primary and standby dumps');
+
+$node_standby_1->stop;
+$node_primary->stop;
diff --git a/src/tools/msvc/vcregress.pl b/src/tools/msvc/vcregress.pl
index bf66083f73..d002bdac9d 100644
--- a/src/tools/msvc/vcregress.pl
+++ b/src/tools/msvc/vcregress.pl
@@ -535,6 +535,8 @@ sub recoverycheck
{
InstallTemp();
+ $ENV{REGRESS_OUTPUTDIR} = "$topdir/src/test/recovery";
+
my $mstat = 0;
my $dir = "$topdir/src/test/recovery";
my $status = tap_check($dir);
--
2.33.1
