On Tue, Jan 18, 2022 at 10:31 AM Kyotaro Horiguchi
<horikyota....@gmail.com> wrote:
>
> At Tue, 18 Jan 2022 12:25:15 +0800, Julien Rouhaud <rjuju...@gmail.com> wrote
> in
> > Hi,
> >
> > On Mon, Jan 17, 2022 at 09:33:57PM +0800, Julien Rouhaud wrote:
> > >
> > > Thanks for the updated patch! Note that thanks to Andres and Thomas
> > > work, you
> > > can now easily rely on the exact same CI than the cfbot on your own github
> > > repository, if you need to debug something on a platform you don't have
> > > access
> > > to. You can check the documentation at [1] for more detail on how to
> > > setup the
> > > CI.
> >
> > The cfbot reports that the patch still fails on Windows but also failed on
> > macos with the same error: https://cirrus-ci.com/task/5655777858813952:
> >
> > [14:20:43.950] # Failed test 'check standby content before timeline
> > switch 0/500FAF8'
> > [14:20:43.950] # at t/003_recovery_targets.pl line 239.
> > [14:20:43.950] # got: '6000'
> > [14:20:43.950] # expected: '7000'
> > [14:20:43.950] # Looks like you failed 1 test of 10.
> >
> > I'm switching the CF entry to Waiting on Author.
>
> The most significant advantages of the local CI setup are
>
> - CI immediately responds to push.
>
> - You can dirty the code with additional logging aid as much as you
> like to see closely what is going on. It makes us hesitant to do
> the same on this ML:p
>
Indeed :)
I found the cause for the test failing on window -- is due to the
custom archive command setting which wasn't setting the correct window
archive directory path.
There is no option to choose a custom wal archive and restore patch in
the TAP test. The attach 0001 patch proposes the same, which enables
you to choose a custom WAL archive and restore directory. The only
concern I have is that $node->info() will print the wrong archive
directory path in that case, do we need to fix that? We might need to
store archive_dir like base_dir, I wasn't sure about that. Thoughts?
Comments?
Regards,
Amul
From 4bc968d3508e9f608fd572f40f39e2cb374d53f4 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Wed, 19 Jan 2022 23:22:02 -0500
Subject: [PATCH v4 1/3] Allow TAP tests to choose custom WAL archive and
restore path.
---
src/test/perl/PostgreSQL/Test/Cluster.pm | 14 ++++++++++----
1 file changed, 10 insertions(+), 4 deletions(-)
diff --git a/src/test/perl/PostgreSQL/Test/Cluster.pm b/src/test/perl/PostgreSQL/Test/Cluster.pm
index b7d4c24553..6a562db2ce 100644
--- a/src/test/perl/PostgreSQL/Test/Cluster.pm
+++ b/src/test/perl/PostgreSQL/Test/Cluster.pm
@@ -1033,10 +1033,13 @@ primary_conninfo='$root_connstr'
# Internal routine to enable archive recovery command on a standby node
sub enable_restoring
{
- my ($self, $root_node, $standby) = @_;
- my $path = PostgreSQL::Test::Utils::perl2host($root_node->archive_dir);
+ my ($self, $root_node, $standby, $path) = @_;
my $name = $self->name;
+ # Default archive directory will be used if not specified.
+ $path = $root_node->archive_dir if (!defined($path));
+ $path = PostgreSQL::Test::Utils::perl2host($path);
+
print "### Enabling WAL restore for node \"$name\"\n";
# On Windows, the path specified in the restore command needs to use
@@ -1101,10 +1104,13 @@ sub set_standby_mode
# Internal routine to enable archiving
sub enable_archiving
{
- my ($self) = @_;
- my $path = PostgreSQL::Test::Utils::perl2host($self->archive_dir);
+ my ($self, $path) = @_;
my $name = $self->name;
+ # Default archive directory will be used if not specified.
+ $path = $self->archive_dir if (!defined($path));
+ $path = PostgreSQL::Test::Utils::perl2host($path);
+
print "### Enabling WAL archiving for node \"$name\"\n";
# On Windows, the path specified in the restore command needs to use
--
2.18.0
From 6c730ee10b9e64d90982aa555283c9094a5fdb87 Mon Sep 17 00:00:00 2001
From: Amul Sul <amul.sul@enterprisedb.com>
Date: Wed, 19 Jan 2022 23:25:07 -0500
Subject: [PATCH v4 2/3] TAP test for EndOfLogTLI
---
src/test/recovery/t/003_recovery_targets.pl | 47 ++++++++++++++++++++-
1 file changed, 46 insertions(+), 1 deletion(-)
diff --git a/src/test/recovery/t/003_recovery_targets.pl b/src/test/recovery/t/003_recovery_targets.pl
index 24da78c0bc..b3f7076540 100644
--- a/src/test/recovery/t/003_recovery_targets.pl
+++ b/src/test/recovery/t/003_recovery_targets.pl
@@ -6,7 +6,7 @@ use strict;
use warnings;
use PostgreSQL::Test::Cluster;
use PostgreSQL::Test::Utils;
-use Test::More tests => 9;
+use Test::More tests => 10;
use Time::HiRes qw(usleep);
# Create and test a standby from given backup, with a certain recovery target.
@@ -182,3 +182,48 @@ $logfile = slurp_file($node_standby->logfile());
ok( $logfile =~
qr/FATAL: .* recovery ended before configured recovery target was reached/,
'recovery end before target reached is a fatal error');
+
+# Test to cover a case where that we are looking for WAL record that ought to be
+# in for e.g 000000010000000000000005 we don't find it; instead we find
+# 000000020000000000000005 because of various reasons such as there was a
+# timeline switch in that segment, and we were reading the old WAL from a
+# segment belonging to a higher timeline or our recovery target timeline is 2,
+# or something that has 2 in its history.
+
+# Insert few more data to primary
+$node_primary->safe_psql('postgres', "SELECT pg_switch_wal()");
+$node_primary->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(6001,7000))");
+my $lsn6 = $node_primary->safe_psql('postgres',
+ "SELECT pg_current_wal_lsn()");
+
+# Setup new standby and enable WAL archiving to archive WAL files at the same
+# location as the primary.
+my $archive_dir = $node_primary->archive_dir;
+$node_standby = PostgreSQL::Test::Cluster->new('standby_9');
+$node_standby->init_from_backup(
+ $node_primary, 'my_backup',
+ has_streaming => 1);
+$node_standby->enable_archiving($archive_dir);
+$node_standby->start;
+# Wait until necessary replay has been done on standby
+$node_primary->wait_for_catchup($node_standby);
+$node_standby->promote;
+$node_standby->safe_psql('postgres',
+ "INSERT INTO tab_int VALUES (generate_series(7001,8000))");
+# Force archiving of WAL file
+my $last_wal = $node_standby->safe_psql('postgres',
+ "SELECT pg_walfile_name(pg_switch_wal())");
+# Wait until this WAL file archive
+my $check_archive = "SELECT last_archived_wal >= '$last_wal' FROM pg_stat_archiver";
+$node_standby->poll_query_until('postgres', $check_archive)
+ or die "Timed out while waiting for $last_wal file archive";
+$node_standby->stop;
+
+# Another standby whose recovery target lsn will be in the WAL file has
+# a different TLI than the target LSN belongs to.
+@recovery_params = (
+ "recovery_target_lsn = '$lsn6'",
+ "recovery_target_inclusive = off");
+test_recovery_standby("target LSN belong to different TLI",
+ 'standby_10', $node_primary, \@recovery_params, "7000", $lsn6);
--
2.18.0
