On 29.09.23 10:02, Peter Eisentraut wrote:
On 26.09.23 17:19, Aleksander Alekseev wrote:
Attached is an updated patch set where I have split the changes into
smaller pieces. The last two patches still have some open questions
about what certain constants mean etc. The other patches should be
settled.
The patches 0001..0005 seem to be ready and rather independent. I
suggest merging them and continue discussing the rest of the patches.
I have committed 0001..0005, and also posted a separate patch to discuss
and correct the behavior of the -c option. I expect that we will carry
over this patch set to the next commit fest.
Here are updated versions of the remaining patches. I took out the
"FIXME" notes about the multipliers applying to the -c option and
replaced them by gentler comments. I don't intend to resolve those
issues here.From 16affb34b3f53b4639a136f9eb1d927c89780ec1 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <pe...@eisentraut.org>
Date: Sun, 29 Oct 2023 16:48:16 +0100
Subject: [PATCH v3 1/2] doc: pg_resetwal: Add comments how the multipliers are
derived
Discussion:
https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507...@eisentraut.org
---
doc/src/sgml/ref/pg_resetwal.sgml | 8 +++++++-
1 file changed, 7 insertions(+), 1 deletion(-)
diff --git a/doc/src/sgml/ref/pg_resetwal.sgml
b/doc/src/sgml/ref/pg_resetwal.sgml
index 08cd3ce5fc2..cf9c7e70f27 100644
--- a/doc/src/sgml/ref/pg_resetwal.sgml
+++ b/doc/src/sgml/ref/pg_resetwal.sgml
@@ -166,7 +166,8 @@ <title>Options</title>
<command>pg_resetwal</command> is unable to determine appropriate values
by reading <filename>pg_control</filename>. Safe values can be determined
as
described below. For values that take numeric arguments, hexadecimal
- values can be specified by using the prefix <literal>0x</literal>.
+ values can be specified by using the prefix <literal>0x</literal>. Note
+ that these instructions only apply with the standard block size of 8 kB.
</para>
<variablelist>
@@ -189,6 +190,7 @@ <title>Options</title>
greatest file name in the same directory. The file names are in
hexadecimal.
</para>
+ <!-- XXX: Should there be a multiplier, similar to the other options? -->
</listitem>
</varlistentry>
@@ -272,6 +274,7 @@ <title>Options</title>
names are in hexadecimal, so the easiest way to do this is to specify
the option value in hexadecimal and append four zeroes.
</para>
+ <!-- 65536 = SLRU_PAGES_PER_SEGMENT * BLCKSZ / sizeof(MultiXactOffset) -->
</listitem>
</varlistentry>
@@ -306,6 +309,7 @@ <title>Options</title>
The file names are in hexadecimal. There is no simple recipe such as
the ones for other options of appending zeroes.
</para>
+ <!-- 52352 = SLRU_PAGES_PER_SEGMENT * floor(BLCKSZ/20) * 4; see
multixact.c -->
</listitem>
</varlistentry>
@@ -354,6 +358,7 @@ <title>Options</title>
in <filename>pg_xact</filename>, <literal>-u 0x700000</literal> will
work (five
trailing zeroes provide the proper multiplier).
</para>
+ <!-- 1048576 = SLRU_PAGES_PER_SEGMENT * BLCKSZ * CLOG_XACTS_PER_BYTE -->
</listitem>
</varlistentry>
@@ -375,6 +380,7 @@ <title>Options</title>
in <filename>pg_xact</filename>, <literal>-x 0x1200000</literal> will
work (five
trailing zeroes provide the proper multiplier).
</para>
+ <!-- 1048576 = SLRU_PAGES_PER_SEGMENT * BLCKSZ * CLOG_XACTS_PER_BYTE -->
</listitem>
</varlistentry>
</variablelist>
base-commit: 237f8765dfd9149471d37f3754d15cef888338a8
--
2.42.0
From 3b5c60dc87f4440aa81d50d0c6363aea7a970347 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <pe...@eisentraut.org>
Date: Sun, 29 Oct 2023 16:48:16 +0100
Subject: [PATCH v3 2/2] pg_resetwal: Add more tests and test coverage
Discussion:
https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507...@eisentraut.org
---
src/bin/pg_resetwal/t/001_basic.pl | 120 +++++++++++++++++++++++++
src/bin/pg_resetwal/t/002_corrupted.pl | 4 +
2 files changed, 124 insertions(+)
diff --git a/src/bin/pg_resetwal/t/001_basic.pl
b/src/bin/pg_resetwal/t/001_basic.pl
index 7e5efbf56b5..6c02c4ae74a 100644
--- a/src/bin/pg_resetwal/t/001_basic.pl
+++ b/src/bin/pg_resetwal/t/001_basic.pl
@@ -14,6 +14,7 @@
my $node = PostgreSQL::Test::Cluster->new('main');
$node->init;
+$node->append_conf('postgresql.conf', 'track_commit_timestamp = on');
command_like([ 'pg_resetwal', '-n', $node->data_dir ],
qr/checkpoint/, 'pg_resetwal -n produces output');
@@ -29,4 +30,123 @@
'check PGDATA permissions');
}
+command_ok([ 'pg_resetwal', '-D', $node->data_dir ], 'pg_resetwal runs');
+$node->start;
+is($node->safe_psql("postgres", "SELECT 1;"), 1, 'server running and working
after reset');
+
+command_fails_like([ 'pg_resetwal', $node->data_dir ], qr/lock file .*
exists/, 'fails if server running');
+
+$node->stop('immediate');
+command_fails_like([ 'pg_resetwal', $node->data_dir ], qr/database server was
not shut down cleanly/, 'does not run after immediate shutdown');
+command_ok([ 'pg_resetwal', '-f', $node->data_dir ], 'runs after immediate
shutdown with force');
+$node->start;
+is($node->safe_psql("postgres", "SELECT 1;"), 1, 'server running and working
after forced reset');
+
+$node->stop;
+
+# check various command-line handling
+
+# Note: This test intends to check that a nonexistent data directory
+# gives a reasonable error message. Because of the way the code is
+# currently structured, you get an error about readings permissions,
+# which is perhaps suboptimal, so feel free to update this test if
+# this gets improved.
+command_fails_like([ 'pg_resetwal', 'foo' ], qr/error: could not read
permissions of directory/, 'fails with nonexistent data directory');
+
+command_fails_like([ 'pg_resetwal', 'foo', 'bar'], qr/too many command-line
arguments/, 'fails with too many command-line arguments');
+
+$ENV{PGDATA} = $node->data_dir; # not used
+command_fails_like([ 'pg_resetwal' ], qr/no data directory specified/, 'fails
with too few command-line arguments');
+
+# error cases
+# -c
+command_fails_like([ 'pg_resetwal', '-c', 'foo', $node->data_dir], qr/error:
invalid argument for option -c/, 'fails with incorrect -c option');
+command_fails_like([ 'pg_resetwal', '-c', '10,bar', $node->data_dir],
qr/error: invalid argument for option -c/, 'fails with incorrect -c option part
2');
+command_fails_like([ 'pg_resetwal', '-c', '1,10', $node->data_dir], qr/greater
than/, 'fails with -c value 1 part 1');
+command_fails_like([ 'pg_resetwal', '-c', '10,1', $node->data_dir], qr/greater
than/, 'fails with -c value 1 part 2');
+# -e
+command_fails_like([ 'pg_resetwal', '-e', 'foo', $node->data_dir], qr/error:
invalid argument for option -e/, 'fails with incorrect -e option');
+command_fails_like([ 'pg_resetwal', '-e', '-1', $node->data_dir], qr/must not
be -1/, 'fails with -e value -1');
+# -l
+command_fails_like([ 'pg_resetwal', '-l', 'foo', $node->data_dir], qr/error:
invalid argument for option -l/, 'fails with incorrect -l option');
+# -m
+command_fails_like([ 'pg_resetwal', '-m', 'foo', $node->data_dir], qr/error:
invalid argument for option -m/, 'fails with incorrect -m option');
+command_fails_like([ 'pg_resetwal', '-m', '10,bar', $node->data_dir],
qr/error: invalid argument for option -m/, 'fails with incorrect -m option part
2');
+command_fails_like([ 'pg_resetwal', '-m', '0,10', $node->data_dir], qr/must
not be 0/, 'fails with -m value 0 part 1');
+command_fails_like([ 'pg_resetwal', '-m', '10,0', $node->data_dir], qr/must
not be 0/, 'fails with -m value 0 part 2');
+# -o
+command_fails_like([ 'pg_resetwal', '-o', 'foo', $node->data_dir], qr/error:
invalid argument for option -o/, 'fails with incorrect -o option');
+command_fails_like([ 'pg_resetwal', '-o', '0', $node->data_dir], qr/must not
be 0/, 'fails with -o value 0');
+# -O
+command_fails_like([ 'pg_resetwal', '-O', 'foo', $node->data_dir], qr/error:
invalid argument for option -O/, 'fails with incorrect -O option');
+command_fails_like([ 'pg_resetwal', '-O', '-1', $node->data_dir], qr/must not
be -1/, 'fails with -O value -1');
+# --wal-segsize
+command_fails_like([ 'pg_resetwal', '--wal-segsize', 'foo', $node->data_dir],
qr/error: invalid value/, 'fails with incorrect --wal-segsize option');
+command_fails_like([ 'pg_resetwal', '--wal-segsize', '13', $node->data_dir],
qr/must be a power/, 'fails with invalid --wal-segsize value');
+# -u
+command_fails_like([ 'pg_resetwal', '-u', 'foo', $node->data_dir], qr/error:
invalid argument for option -u/, 'fails with incorrect -u option');
+command_fails_like([ 'pg_resetwal', '-u', '1', $node->data_dir], qr/must be
greater than/, 'fails with -u value too small');
+# -x
+command_fails_like([ 'pg_resetwal', '-x', 'foo', $node->data_dir], qr/error:
invalid argument for option -x/, 'fails with incorrect -x option');
+command_fails_like([ 'pg_resetwal', '-x', '1', $node->data_dir], qr/must be
greater than/, 'fails with -x value too small');
+
+# run with control override options
+
+my $out = (run_command([ 'pg_resetwal', '-n', $node->data_dir ]))[0];
+$out =~ /^Database block size: *(\d+)$/m or die;
+my $blcksz = $1;
+
+my @cmd = ('pg_resetwal', '-D', $node->data_dir);
+
+# some not-so-critical hardcoded values
+push @cmd, '-e', 1;
+push @cmd, '-l', '00000001000000320000004B';
+push @cmd, '-o', 100_000;
+push @cmd, '--wal-segsize', 1;
+
+# these use the guidance from the documentation
+
+sub get_slru_files
+{
+ opendir(my $dh, $node->data_dir . '/' . $_[0]) or die $!;
+ my @files = sort grep { /[0-9A-F]+/ } readdir $dh;
+ closedir $dh;
+ return @files;
+}
+
+my (@files, $mult);
+
+@files = get_slru_files('pg_commit_ts');
+# XXX: Should there be a multiplier, similar to the other options?
+# -c argument is "old,new"
+push @cmd,
+ '-c', sprintf("%d,%d",
+ hex($files[0]) == 0 ? 3 : hex($files[0]), hex($files[-1]));
+
+@files = get_slru_files('pg_multixact/offsets');
+$mult = 32 * $blcksz / 4;
+# -m argument is "new,old"
+push @cmd,
+ '-m', sprintf("%d,%d",
+ (hex($files[-1]) + 1) * $mult,
+ hex($files[0]) == 0 ? 1 : hex($files[0] * $mult));
+
+@files = get_slru_files('pg_multixact/members');
+$mult = 32 * int($blcksz/20) * 4;
+push @cmd,
+ '-O', (hex($files[-1]) + 1) * $mult;
+
+@files = get_slru_files('pg_xact');
+$mult = 32 * $blcksz * 4;
+push @cmd,
+ '-u', (hex($files[0]) == 0 ? 3 : hex($files[0]) * $mult),
+ '-x', ((hex($files[-1]) + 1) * $mult);
+
+command_ok([@cmd, '-n'], 'runs with control override options, dry run');
+command_ok(\@cmd, 'runs with control override options');
+command_like([ 'pg_resetwal', '-n', $node->data_dir ], qr/^Latest checkpoint's
NextOID: *100000$/m, 'spot check that control changes were applied');
+
+$node->start;
+ok(1, 'server started after reset');
+
done_testing();
diff --git a/src/bin/pg_resetwal/t/002_corrupted.pl
b/src/bin/pg_resetwal/t/002_corrupted.pl
index b3a37728a42..ee888504730 100644
--- a/src/bin/pg_resetwal/t/002_corrupted.pl
+++ b/src/bin/pg_resetwal/t/002_corrupted.pl
@@ -55,4 +55,8 @@
],
'processes zero WAL segment size');
+# now try to run it
+command_fails_like([ 'pg_resetwal', $node->data_dir ], qr/not proceeding
because control file values were guessed/, 'does not run when control file
values were guessed');
+command_ok([ 'pg_resetwal', '-f', $node->data_dir ], 'runs with force when
control file values were guessed');
+
done_testing();
--
2.42.0
