While looking into issues with fairywren and pg_basebackup tests, I
created a similar environment but with more modern Windows / msys2.
Before it even got to the test that failed on fairywren it failed the
first TAP test for a variety of reasons, all connected to
TestLib::perl2host.
First, this function is in some cases returning paths for directories
with trailing slashes and or embedded double slashes. Both of these can
cause problems, especially when written to a tablespace map file. Also,
the cygpath invocation is returning a path with backslashes whereas "pwd
-W' returns a path with forward slashes.
So the first attached patch rectifies these problems. It fixes issues
with doubles and trailing slashes and makes cygpath return a path with
forward slashes just like the non-cygpath branch.
However, there is another problem, which is that if called on a path
that includes a symlink, on the test platform I set up it actually
resolves that link rather than just following it. The end result is that
the use of a shorter path via a symlink is effectively defeated. I
haven't found any way to stop this behaviour.
The second patch therefore adjusts the test to avoid calling perl2host
on such a path. It just calls perl2host on the symlink's parent, and
thereafter uses that result.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
>From eee31e6fd8f183ae0d836dfc131537912979d954 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <and...@dunslane.net>
Date: Mon, 26 Jul 2021 09:34:59 -0400
Subject: [PATCH 1/2] Make TestLib::perl2host more consistent and robust
Sometimes cygpath has been observed to return a path with a trailing
slash. That can cause problems, Also, make "cygpath" usage
consistent with "pwd -W" with respect to the use of forward slashes.
Backpatch to release 14 where the current code was introduced.
---
src/test/perl/TestLib.pm | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/test/perl/TestLib.pm b/src/test/perl/TestLib.pm
index 15572abbea..cbab1587cc 100644
--- a/src/test/perl/TestLib.pm
+++ b/src/test/perl/TestLib.pm
@@ -304,6 +304,8 @@ except for the case of Perl=msys and host=mingw32. The subject need not
exist, but its parent or grandparent directory must exist unless cygpath is
available.
+The returned path uses forward slashes but has no trailing slash.
+
=cut
sub perl2host
@@ -313,10 +315,11 @@ sub perl2host
if ($is_msys2)
{
# get absolute, windows type path
- my $path = qx{cygpath -a -w "$subject"};
+ my $path = qx{cygpath -a -m "$subject"};
if (!$?)
{
chomp $path;
+ $path =~ s!/$!!;
return $path if $path;
}
# fall through if this didn't work.
@@ -342,6 +345,7 @@ sub perl2host
# this odd way of calling 'pwd -W' is the only way that seems to work.
my $dir = qx{sh -c "pwd -W"};
chomp $dir;
+ $dir =~ s!/$!!;
chdir $here;
return $dir . $leaf;
}
--
2.25.4
>From c50c319df2fba24779861f410c5dda8303e80a28 Mon Sep 17 00:00:00 2001
From: Andrew Dunstan <and...@dunslane.net>
Date: Wed, 28 Jul 2021 07:29:38 -0400
Subject: [PATCH 2/2] Avoid calling TestLib::perl2host on a symlinked directory
Certain versions of msys2/Windows have been observed to resolve symlinks
in perl2host rather than just follow them. This defeats using a
symlinked shorter path to a longer path, and makes certain tests fail.
We therefore call perl2host on the parent directory of the symlink and
thereafter just use that result.
Apply to release 14 where the problem has been observed.
---
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 12 +++++++-----
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/src/bin/pg_basebackup/t/010_pg_basebackup.pl b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
index 74f8c2c739..bde31b3c03 100644
--- a/src/bin/pg_basebackup/t/010_pg_basebackup.pl
+++ b/src/bin/pg_basebackup/t/010_pg_basebackup.pl
@@ -238,11 +238,13 @@ $node->start;
# to our physical temp location. That way we can use shorter names
# for the tablespace directories, which hopefully won't run afoul of
# the 99 character length limit.
-my $shorter_tempdir = TestLib::tempdir_short . "/tempdir";
+my $sys_tempdir = TestLib::tempdir_short;
+my $real_sys_tempdir = TestLib::perl2host($sys_tempdir) . "/tempdir";
+my $shorter_tempdir = $sys_tempdir . "/tempdir";
dir_symlink "$tempdir", $shorter_tempdir;
mkdir "$tempdir/tblspc1";
-my $realTsDir = TestLib::perl2host("$shorter_tempdir/tblspc1");
+my $realTsDir = "$real_sys_tempdir/tblspc1";
my $real_tempdir = TestLib::perl2host($tempdir);
$node->safe_psql('postgres',
"CREATE TABLESPACE tblspc1 LOCATION '$realTsDir';");
@@ -275,7 +277,7 @@ SKIP:
# Recover tablespace into a new directory (not where it was!)
my $repTsDir = "$tempdir/tblspc1replica";
- my $realRepTsDir = TestLib::perl2host("$shorter_tempdir/tblspc1replica");
+ my $realRepTsDir = "$real_sys_tempdir/tblspc1replica";
mkdir $repTsDir;
TestLib::system_or_bail($tar, 'xf', $tblspc_tars[0], '-C', $repTsDir);
@@ -390,7 +392,7 @@ ok( -d "$tempdir/backup1/pg_replslot",
rmtree("$tempdir/backup1");
mkdir "$tempdir/tbl=spc2";
-$realTsDir = TestLib::perl2host("$shorter_tempdir/tbl=spc2");
+$realTsDir = "$real_sys_tempdir/tbl=spc2";
$node->safe_psql('postgres', "DROP TABLE test1;");
$node->safe_psql('postgres', "DROP TABLE tblspc1_unlogged;");
$node->safe_psql('postgres', "DROP TABLESPACE tblspc1;");
@@ -409,7 +411,7 @@ $node->safe_psql('postgres', "DROP TABLESPACE tblspc2;");
rmtree("$tempdir/backup3");
mkdir "$tempdir/$superlongname";
-$realTsDir = TestLib::perl2host("$shorter_tempdir/$superlongname");
+$realTsDir = "$real_sys_tempdir/$superlongname";
$node->safe_psql('postgres',
"CREATE TABLESPACE tblspc3 LOCATION '$realTsDir';");
$node->command_ok([ 'pg_basebackup', '-D', "$tempdir/tarbackup_l3", '-Ft' ],
--
2.25.4
