Hello Thomas,
10.11.2023 06:31, Thomas Munro wrote:
Here is a new attempt to fix this mess. Disclaimer: this based
entirely on reading the manual and vicariously hacking a computer I
don't have via CI.
As it also might (and I would like it to) be the final attempt, I decided
to gather information and all the cases that we had on this topic.
At least, for the last 5 years we've seen:
[1] 2019-01-18: Re: BUG #15598: PostgreSQL Error Code is not reported when connection terminated due to
idle-in-transaction timeout
test 099_case_15598 made (in attachment)
no commit
[2] 2019-01-22: Rare SSL failures on eelpout
references [1]
test 099_rare_ssl_failures made (in attachment)
commit 2019-03-19 1f39a1c06: Restructure libpq's hqandling of send failures.
[3] 2019-11-28: pgsql: Add tests for '-f' option in dropdb utility.
test 051_dropdb_force was proposed (included in the attachment)
commit 2019-11-30 98a9b37ba: Revert commits 290acac92b and 8a7e9e9dad.
[4] 2019-12-06: closesocket behavior in different platforms
references [1], [2], [3]; a documentation change proposed
no commit
[5] 2020-06-03: libpq copy error handling busted
test 099_pgbench_with_server_off made (in attachment)
commit 2020-06-07 7247e243a: Try to read data from the socket in pqSendSome's
write_failed paths. (a fix for 1f39a1c06)
[6] 2020-10-19: BUG #16678: The ecpg connect/test5 test sometimes fails on
Windows
no commit
[7] 2021-11-17: Windows: Wrong error message at connection termination
references [6]
commit: 2021-12-02 6051857fc: On Windows, close the client socket explicitly
during backend shutdown.
[8] 2021-12-05 17:03:18: MSVC SSL test failure
commit: 2021-12-07 ed52c3707: On Windows, also call shutdown() while closing
the client socket.
[9] 2021-12-30: Why is src/test/modules/committs/t/002_standby.pl flaky?
additional test 099_postgres_fdw_disconnect made (in attachment)
commit 2022-01-26 75674c7ec: Revert "graceful shutdown" changes for Windows, in
back branches only. (REL_14_STABLE)
commit 2022-03-22 29992a6a5: Revert "graceful shutdown" changes for Windows.
(master)
[10] 2022-02-02 19:19:22: BUG #17391: While using --with-ssl=openssl and PG_TEST_EXTRA='ssl' options, SSL tests fail on
OpenBSD 7.0
commit 2022-02-12 335fa5a26: Fix thinko in PQisBusy(). (a fix for 1f39a1c06)
commit 2022-02-12 faa189c93: Move libpq's write_failed mechanism down to
pqsecure_raw_write(). (a fix for 1f39a1c06)
As it becomes difficult to test/check all those cases scattered around, I
decided to retell the whole story by means of tests. Please look at the
script win-sock-tests-01.cmd attached, which can be executed both on
Windows (as regular cmd) and on Linux (bash win-sock-tests-01.cmd).
At the end of the script we can see several things.
First, the last patchset you posted, applied to b282fa88d~1, fixes the
issue discussed in this thread (it eliminates failures of
commit_ts_002_standby (and also 099_postgres_fdw_disconnect)).
Second, with the sleep added (see [6]), I had the same results of
`meson test` on Windows and on Linux.
Namely, there are some tests failing (see win-sock-tests-01.cmd) due to
walsender preventing server stop.
I describe this issue separately (see details in walsender-cannot-exit.txt;
maybe it's worth to discuss it in a separate thread) as it's kind of
off-topic. With the supplementary sleep() added to WalSndLoop(), the
complete `meson test` passes successfully both on Windows and on Linux.
Third, cases [1] and [3] are still broken, due to a Windows peculiarity.
Please see server.c and client.c attached, which demonstrate:
Case "no shutdown/closesocket" on Windows:
C:\src>server
Listening for incoming connections...
C:\src>client
Client connected: 127.0.0.1:64395
Connection to server established. Enter message: msg
Client message: msg
Sending message...
Sleeping...
Exiting...
C:\src>
Calling recv()...
recv() failed
Case "no shutdown/closesocket" on Linux:
$ server
Listening for incoming connections...
$ client
Client connected: 127.0.0.1:33044
Connection to server established. Enter message: msg
Client message: msg
Sending message...
Sleeping...
Exiting...
$
Calling recv()...
Server message: MESSAGE
Case "shutdown/closesocket" on Windows:
C:\src>server shutdown closesocket
Listening for incoming connections...
C:\src>client
Client connected: 127.0.0.1:64396
Connection to server established. Enter message: msg
Client message: msg
Sending message...
Sleeping...
Calling shutdown()...
Calling closesocket()...
Exiting...
C:\src>
Calling recv()...
Server message: MESSAGE
That's okay so far, but what makes cases [1]/[3] different from all cases
in the whole existing test suite, which now performed successfully, is
that psql calls send() before recv() on a socket closed and abandoned by
the server.
Those programs show on Windows:
C:\src>server shutdown closesocket
Listening for incoming connections...
C:\src>client send_before_recv
Client connected: 127.0.0.1:64397
Connection to server established. Enter message: msg
Client message: msg
Sending message...
Sleeping...
Calling shutdown()...
Calling closesocket()...
Exiting...
C:\src>
send() returned 4
Calling recv()...
recv() failed
As known, on Linux the same scenario works just fine.
Fourth, tests 099_rare_ssl_failures (and 001_ssl_tests, though more rarely)
fail for me with the latest patches (only on Windows again):
...
8/10 postgresql:ssl_1 / ssl_1/099_rare_ssl_failures ERROR 141.34s
exit status 3
...
9/10 postgresql:ssl_7 / ssl_7/099_rare_ssl_failures OK 142.52s
2000 subtests passed
10/10 postgresql:ssl_6 / ssl_6/099_rare_ssl_failures OK 143.00s
2000 subtests passed
Ok: 2
Expected Fail: 0
Fail: 8
ssl_1\099_rare_ssl_failures\log\regress_log_099_rare_ssl_failures.txt:
...
iteration 354
[20:57:06.984](0.106s) ok 707 - certificate authorization fails with revoked
client cert with server-side CRL directory
[20:57:06.984](0.000s) ok 708 - certificate authorization fails with revoked client cert with server-side CRL directory:
matches
iteration 355
[20:57:07.156](0.172s) ok 709 - certificate authorization fails with revoked
client cert with server-side CRL directory
[20:57:07.156](0.001s) not ok 710 - certificate authorization fails with revoked client cert with server-side CRL
directory: matches
[20:57:07.159](0.003s) # Failed test 'certificate authorization fails with revoked client cert with server-side CRL
directory: matches'
# at .../src/test/ssl_1/t/099_rare_ssl_failures.pl line 88.
[20:57:07.159](0.000s) # 'psql: error: connection to server at "127.0.0.1", port 59843 failed: could
not receive data from server: Software caused connection abort (0x00002745/10053)
# SSL SYSCALL error: Software caused connection abort (0x00002745/10053)'
# doesn't match '(?^:SSL error: sslv3 alert certificate revoked)'
...
It seems to me that it can have the same explanation (if openssl can call
send() before recv() under the hood), but maybe it should be investigated
further.
Review/poking-with-a-stick/trying-to-break-it most welcome.
I could not find anything suspicious in the code, except for maybe a typo
"The are ...".
[1]
https://www.postgresql.org/message-id/flat/87k1iy44fd.fsf%40news-spur.riddles.org.uk#ba0c07f13c300d42fd537855dd95dd2b
[2]
https://www.postgresql.org/message-id/flat/CAEepm%3D2n6Nv%2B5tFfe8YnkUm1fXgvxR0Mm1FoD%2BQKG-vLNGLyKg%40mail.gmail.com
[3]
https://www.postgresql.org/message-id/flat/e1iad8h-0004us...@gemulon.postgresql.org
[4]
https://www.postgresql.org/message-id/flat/CALDaNm2tEvr_Kum7SyvFn0%3D6H3P0P-Zkhnd%3DdkkX%2BQ%3DwKutZ%3DA%40mail.gmail.com
[5]
https://www.postgresql.org/message-id/flat/20200603201242.ofvm4jztpqytwfye%40alap3.anarazel.de
[6] https://www.postgresql.org/message-id/16678-253e48d34dc0c...@postgresql.org
[7]
https://www.postgresql.org/message-id/flat/90b34057-4176-7bb0-0dbb-9822a5f6425b%40greiz-reinsdorf.de
[8]
https://www.postgresql.org/message-id/flat/af5e0bf3-6a61-bb97-6cba-061ddf22ff6b%40dunslane.net
[9]
https://www.postgresql.org/message-id/flat/CA%2BhUKG%2BOeoETZQ%3DQw5Ub5h3tmwQhBmDA%3DnuNO3KG%3DzWfUypFAw%40mail.gmail.com
[10]
https://www.postgresql.org/message-id/flat/17391-304f81bcf724b58b%40postgresql.org
Best regards,
Alexander
With pg_usleep(10000L) added in pqReadData() you can see several tests
failed when running make check-world/meson test.
For example:
meson test recovery/019_replslot_limit
results in:
[16:25:18.510](6.771s) ok 16 - check if checkpoint command is not blocked
### Stopping node "primary2" using mode fast
# Running: pg_ctl -D
.../build/testrun/recovery/019_replslot_limit/data/t_019_replslot_limit_primary2_data/pgdata
-m fast stop
waiting for server to shut
down............................................................... failed
pg_ctl: server does not shut down
# pg_ctl stop failed: 256
# Postmaster PID for node "primary2" is 209253
[16:26:18.567](60.057s) Bail out! pg_ctl stop failed
As I understand, it happens because WalSndLoop() performs this in a loop:
...
if (got_SIGUSR2)
WalSndDone(send_data);
...
/* Sleep until something happens or we time out */
WalSndWait(wakeEvents, sleeptime,
WAIT_EVENT_WAL_SENDER_MAIN);
...
where WalSndDone() does this:
replicatedPtr = XLogRecPtrIsInvalid(MyWalSnd->flush) ?
MyWalSnd->write : MyWalSnd->flush;
if (WalSndCaughtUp && sentPtr == replicatedPtr &&
!pq_is_send_pending())
{
...
proc_exit(0);
}
if (!waiting_for_ping_response)
WalSndKeepalive(true, InvalidXLogRecPtr);
(WalSndWait's call chain is:
WalSndWait -> WaitEventSetWait -> WaitEventSetWaitBlock -> epoll_wait)
So, WalSndDone waits here for replicatedPtr to become equal to sentPtr.
In other words, walsender sends keepalive to walreceiver and waits for a
new flush position from it.
And the standby loop is as follows:
WalReceiverMain:
for (;;)
{
if (len > 0)
{
...
XLogWalRcvProcessMsg(buf[0], &buf[1], len - 1,
startpointTLI);
...
}
else if (len == 0)
break;
}
...
XLogWalRcvFlush(false, startpointTLI);
where XLogWalRcvProcessMsg() calls XLogWalRcvSendReply() if a keepalive
message received.
So if walreceiver constantly receives some messages (keepalive messages in
this case), it just can't get out of the loop and perform XLogWalRcvFlush()
/move Flush position what primary is waiting for.
But if something delays walsender for more than 10 ms between epoll_wait()
and WalSndDone() on a next iteration, standby manages to get len = 0 and to
perform Flush.
#include <stdio.h>
#include <string.h>
#if defined(_WIN32)
#include <winsock.h>
#pragma comment(lib, "Ws2_32.lib")
#define SD_SEND 1
#else
#include <sys/socket.h>
#include <arpa/inet.h>
#endif
int main(int argc, char *argv[])
{
char *server_message = "MESSAGE";
char client_message[2000] = { 0 };
#if defined(_WIN32)
WSADATA wsaData;
int iResult = WSAStartup(MAKEWORD(2,2), &wsaData);
if (iResult != NO_ERROR) { printf("WSAStartup failed: %d\n", iResult); return 1; }
#endif
int socket_desc = socket(AF_INET, SOCK_STREAM, 0);
if (socket_desc < 0) {printf("Could not create socket\n"); return 1;}
struct sockaddr_in server_addr;
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(2000);
server_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
if (bind(socket_desc, (struct sockaddr *)&server_addr, sizeof(server_addr)) < 0) { printf("bind() failed\n"); return 1; }
if (listen(socket_desc, 1) < 0) { printf("listen() failed\n"); return 1; }
printf("Listening for incoming connections...\n");
struct sockaddr_in client_addr;
int client_size = sizeof(client_addr);
int client_sock = accept(socket_desc, (struct sockaddr*)&client_addr, &client_size);
if (client_sock < 0) { printf("accept() failed\n"); return 1; }
printf("Client connected: %s:%i\n", inet_ntoa(client_addr.sin_addr), ntohs(client_addr.sin_port));
if (recv(client_sock, client_message, sizeof(client_message), 0) < 0) { printf("recv() failed\n"); return -1; }
printf("Client message: %s\n", client_message);
printf("Sending message...\n");
if (send(client_sock, server_message, strlen(server_message), 0) < 0) { printf("send() failed\n"); return 1; }
#if defined(_WIN32)
if (argc > 1 && strcmp(argv[1], "shutdown") == 0)
{
printf("Calling shutdown()...\n");
shutdown(client_sock, SD_SEND);
}
if ((argc > 1 && strcmp(argv[argc - 1], "closesocket") == 0))
{
printf("Calling closesocket()...\n");
closesocket(client_sock);
}
#endif
printf("Exiting...\n");
return 0;
}
#include <stdio.h>
#include <string.h>
#if defined(_WIN32)
#include <windows.h>
#include <winsock.h>
#pragma comment (lib, "Ws2_32.lib")
#define sleep(s) Sleep((s) * 1000)
#else
#include <sys/socket.h>
#include <arpa/inet.h>
#include <unistd.h>
#endif
int main(int argc, char *argv[])
{
char server_message[2000] = { 0 };
char client_message[2000] = { 0 };
#if defined(_WIN32)
WSADATA wsaData;
int iResult = WSAStartup(MAKEWORD(2,2), &wsaData);
if (iResult != NO_ERROR) { printf("WSAStartup failed: %d\n", iResult); return 1; }
#endif
int socket_desc = socket(AF_INET, SOCK_STREAM, 0);
if (socket_desc < 0) { printf("Could not create socket\n"); return -1; }
struct sockaddr_in server_addr;
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(2000);
server_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
if (connect(socket_desc, (struct sockaddr*)&server_addr, sizeof(server_addr)) < 0) { printf("Could not connect\n"); return -1; }
printf("Connection to server established. Enter message: ");
fgets(client_message, sizeof(client_message), stdin);
if (send(socket_desc, client_message, strlen(client_message), 0) < 0) { printf("send() failed\n"); return -1; }
printf("Sleeping...\n");
sleep(3);
if (argc > 1 && strcmp(argv[1], "send_before_recv") == 0)
{
int r = send(socket_desc, client_message, strlen(client_message), 0);
printf("send() returned %d\n", r);
}
printf("Calling recv()...\n");
if (recv(socket_desc, server_message, sizeof(server_message), 0) < 0) { printf("recv() failed\n"); return 1; }
printf("Server message: %s\n", server_message);
return 0;
}
diff --git a/contrib/postgres_fdw/meson.build b/contrib/postgres_fdw/meson.build
index 2b451f165e..674ab0a7b9 100644
--- a/contrib/postgres_fdw/meson.build
+++ b/contrib/postgres_fdw/meson.build
@@ -39,4 +39,9 @@ tests += {
],
'regress_args': ['--dlpath', meson.build_root() / 'src/test/regress'],
},
+ 'tap': {
+ 'tests': [
+ 't/099_postgres_fdw_disconnect.pl',
+ ],
+ },
}
diff --git a/contrib/postgres_fdw/t/099_postgres_fdw_disconnect.pl b/contrib/postgres_fdw/t/099_postgres_fdw_disconnect.pl
new file mode 100644
index 0000000000..f1b51fb448
--- /dev/null
+++ b/contrib/postgres_fdw/t/099_postgres_fdw_disconnect.pl
@@ -0,0 +1,87 @@
+# Test postgres_fdw reaction on target server disconnection
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Utils;
+use Test::More tests => 1;
+use PostgreSQL::Test::Cluster;
+
+use Time::HiRes qw(time usleep);
+use DateTime;
+
+use threads;
+use threads::shared;
+
+sub restart_thread {
+ my @args = @_;
+ my ($server) = @args;
+ $SIG{'KILL'} = sub { threads->exit(); };
+ sleep(3);
+ for (my $i = 0; $i < 100; $i++) {
+ sleep(5);
+ $server->restart();
+ }
+}
+
+my $foreign = PostgreSQL::Test::Cluster->new('foreign');
+$foreign->init();
+
+$foreign->append_conf(
+ 'postgresql.conf', qq{
+log_min_messages = DEBUG1
+log_min_error_statement = log
+log_connections = on
+log_disconnections = on
+log_line_prefix = '%m|%u|%d|%c|'
+log_statement = 'all'
+ });
+$foreign->start;
+my $foreign_port = $foreign->port;
+
+my $local = PostgreSQL::Test::Cluster->new('local');
+$local->init();
+
+$local->append_conf(
+ 'postgresql.conf', qq{
+log_min_messages = DEBUG1
+log_min_error_statement = log
+log_connections = on
+log_disconnections = on
+log_line_prefix = '%m|%u|%d|%c|'
+log_statement = 'all'
+ });
+$local->start();
+
+$foreign->safe_psql('postgres', qq{
+CREATE TABLE large(a int, t text);
+INSERT INTO large SELECT x, rpad(x::text, 100) FROM generate_series(0, 999999) x;
+});
+
+
+$local->safe_psql('postgres', qq{
+CREATE EXTENSION postgres_fdw;
+CREATE SERVER fpg FOREIGN DATA WRAPPER postgres_fdw OPTIONS (dbname 'postgres', port '$foreign_port');
+CREATE USER MAPPING FOR CURRENT_USER SERVER fpg;
+CREATE FUNCTION fx2(i integer) RETURNS int AS 'begin return i * 2; end;' LANGUAGE plpgsql;
+});
+
+my $outputdir = $PostgreSQL::Test::Utils::tmp_check;
+
+$local->safe_psql('postgres', 'IMPORT FOREIGN SCHEMA public FROM SERVER fpg INTO public;');
+print($local->psql('postgres', 'EXPLAIN (VERBOSE) SELECT * FROM large WHERE a = fx2(a)') . "\n");
+my $thread = threads->create('restart_thread', $foreign);
+$ENV{PGHOST} = $local->host;
+$ENV{PGPORT} = $local->port;
+$ENV{PGCTLTIMEOUT} = 180;
+for (my $i = 1; $i <= 50; $i++) {
+ diag(" executing query ($i)...");
+# Avoid using IPC::Run due to it's own quirks
+# my ($ret, $stdout, $stderr) = $local->psql('postgres', 'SELECT * FROM large WHERE a = fx2(a)');
+ my $ret = system("psql postgres -c \"SELECT $i i, * FROM large WHERE a = fx2(a)\" >>\"$outputdir/psql.log\" 2>&1");
+ diag(" result: \t$ret");
+}
+$thread->kill('KILL')->detach;
+sleep(2);
+$foreign->_update_pid(1);
+ok(1);
diff --git a/src/test/modules/test_misc/meson.build b/src/test/modules/test_misc/meson.build
index 911084ac0f..bee227eabe 100644
--- a/src/test/modules/test_misc/meson.build
+++ b/src/test/modules/test_misc/meson.build
@@ -10,6 +10,11 @@ tests += {
't/002_tablespace.pl',
't/003_check_guc.pl',
't/004_io_direct.pl',
+ 't/099_case_15598.pl',
+ 't/051_dropdb_force.pl',
+ 't/099_pgbench_with_server_off.pl',
+ 't/099_ecpg_16678.pl',
+ 't/099_commit_ts_002_standby.pl',
],
},
}
diff --git a/src/test/modules/test_misc/t/051_dropdb_force.pl b/src/test/modules/test_misc/t/051_dropdb_force.pl
new file mode 100644
index 0000000000..94d170f081
--- /dev/null
+++ b/src/test/modules/test_misc/t/051_dropdb_force.pl
@@ -0,0 +1,106 @@
+#
+# Tests the force option of drop database command.
+#
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+# To avoid hanging while expecting some specific input from a psql
+# instance being driven by us, add a timeout high enough that it
+# should never trigger even on very slow machines, unless something
+# is really wrong.
+my $psql_timeout = IPC::Run::timer(60);
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+$node->init;
+$node->start;
+
+# Create a database that will be dropped. This will test that the force
+# option works when no other backend is connected to the database being
+# dropped.
+$node->safe_psql('postgres', 'CREATE DATABASE foobar');
+$node->issues_sql_like(
+ [ 'dropdb', '--force', 'foobar' ],
+ qr/statement: DROP DATABASE foobar WITH \(FORCE\);/,
+ 'SQL DROP DATABASE (FORCE) run');
+
+# database foobar must not exist.
+is( $node->safe_psql(
+ 'postgres',
+ qq[SELECT EXISTS(SELECT * FROM pg_database WHERE datname='foobar');]
+ ),
+ 'f',
+ 'database foobar was removed');
+
+# Create a database that will be dropped. This will test that the force
+# option works when one other backend is connected to the database being
+# dropped.
+$node->safe_psql('postgres', 'CREATE DATABASE foobar1');
+
+# Run psql, keeping session alive, so we have an alive backend to kill.
+my ($killme_stdin, $killme_stdout, $killme_stderr) = ('', '', '');
+my $killme = IPC::Run::start(
+ [
+ 'psql', '-X', '-qAt', '-v', 'ON_ERROR_STOP=1', '-f', '-', '-d',
+ $node->connstr('foobar1')
+ ],
+ '<',
+ \$killme_stdin,
+ '>',
+ \$killme_stdout,
+ '2>',
+ \$killme_stderr,
+ $psql_timeout);
+
+# Ensure killme process is active.
+$killme_stdin .= q[
+SELECT pg_backend_pid();
+];
+ok( pump_until(
+ $killme, $psql_timeout, \$killme_stdout, qr/[[:digit:]]+[\r\n]$/m),
+ 'acquired pid for SIGTERM');
+my $pid = $killme_stdout;
+chomp($pid);
+$killme_stdout = '';
+$killme_stderr = '';
+
+# Check the connections on foobar1 database.
+is( $node->safe_psql(
+ 'postgres',
+ qq[SELECT pid FROM pg_stat_activity WHERE datname='foobar1' AND pid = $pid;]
+ ),
+ $pid,
+ 'database foobar1 is used');
+
+# Now drop database with dropdb --force command.
+$node->issues_sql_like(
+ [ 'dropdb', '--force', 'foobar1' ],
+ qr/statement: DROP DATABASE foobar1 WITH \(FORCE\);/,
+ 'SQL DROP DATABASE (FORCE) run');
+
+# Check that psql sees the killed backend as having been terminated.
+$killme_stdin .= q[
+SELECT 1;
+];
+ok( pump_until(
+ $killme, $psql_timeout, \$killme_stderr,
+ qr/FATAL: terminating connection due to administrator command/m),
+ "psql query died successfully after SIGTERM");
+$killme_stderr = '';
+$killme_stdout = '';
+$killme->finish;
+
+# database foobar1 must not exist.
+is( $node->safe_psql(
+ 'postgres',
+ qq[SELECT EXISTS(SELECT * FROM pg_database WHERE datname='foobar1');]
+ ),
+ 'f',
+ 'database foobar1 was removed');
+
+$node->stop();
+
+done_testing();
diff --git a/src/test/modules/test_misc/t/099_case_15598.pl b/src/test/modules/test_misc/t/099_case_15598.pl
new file mode 100644
index 0000000000..57243b04d7
--- /dev/null
+++ b/src/test/modules/test_misc/t/099_case_15598.pl
@@ -0,0 +1,61 @@
+# Run ecpg regression tests in a loop
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+use File::Basename;
+
+# Initialize node
+my $node = PostgreSQL::Test::Cluster->new('p1');
+$node->init(
+);
+
+$node->append_conf(
+ 'postgresql.conf',
+ qq{
+});
+
+$node->start;
+
+$node->safe_psql('postgres', 'select 1');
+
+my $psql_timeout = IPC::Run::timer(10);
+# Run psql, ...
+my ($psql_stdin, $psql_stdout, $psql_stderr) = ('', '', '');
+my $psql = IPC::Run::start(
+ [
+ 'psql', '-X', '-At', '-v', 'ON_ERROR_STOP=1', '-f', '-', '-d',
+ $node->connstr('postgres')
+ ],
+ '<',
+ \$psql_stdin,
+ '>',
+ \$psql_stdout,
+ '2>',
+ \$psql_stderr,
+ $psql_timeout);
+
+$psql_stdin .= q[
+SET idle_in_transaction_session_timeout=500;
+BEGIN;
+SELECT * FROM pg_class;
+];
+
+$psql->pump_nb();
+sleep(1);
+
+$psql_stdin .= q[
+SELECT 'OK';
+];
+
+ok(pump_until(
+ $psql, $psql_timeout, \$psql_stderr, qr/FATAL: terminating connection due to idle-in-transaction timeout/m), "expected FATAL message received");
+
+print($psql_stdin . "\n");
+print("psql stdout: ". $psql_stdout . "\npsql stderr: " . $psql_stderr. "\n");
+
+ok(1);
+$node->stop;
+
+done_testing();
diff --git a/src/test/modules/test_misc/t/099_commit_ts_002_standby.pl b/src/test/modules/test_misc/t/099_commit_ts_002_standby.pl
new file mode 100644
index 0000000000..59cc2b1244
--- /dev/null
+++ b/src/test/modules/test_misc/t/099_commit_ts_002_standby.pl
@@ -0,0 +1,68 @@
+
+# Copyright (c) 2021-2023, PostgreSQL Global Development Group
+
+# Test simple scenario involving a standby
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Utils;
+use Test::More;
+use PostgreSQL::Test::Cluster;
+
+my $bkplabel = 'backup';
+my $primary = PostgreSQL::Test::Cluster->new('primary');
+$primary->init(allows_streaming => 1);
+
+$primary->append_conf(
+ 'postgresql.conf', qq{
+ track_commit_timestamp = on
+ max_wal_senders = 5
+ });
+$primary->start;
+$primary->backup($bkplabel);
+
+my $standby = PostgreSQL::Test::Cluster->new('standby');
+$standby->init_from_backup($primary, $bkplabel, has_streaming => 1);
+$standby->start;
+
+for my $i (1 .. 10)
+{
+ $primary->safe_psql('postgres', "create table t$i()");
+}
+my $primary_ts = $primary->safe_psql('postgres',
+ qq{SELECT ts.* FROM pg_class, pg_xact_commit_timestamp(xmin) AS ts WHERE relname = 't10'}
+);
+my $primary_lsn =
+ $primary->safe_psql('postgres', 'select pg_current_wal_lsn()');
+$standby->poll_query_until('postgres',
+ qq{SELECT '$primary_lsn'::pg_lsn <= pg_last_wal_replay_lsn()})
+ or die "standby never caught up";
+
+my $standby_ts = $standby->safe_psql('postgres',
+ qq{select ts.* from pg_class, pg_xact_commit_timestamp(xmin) ts where relname = 't10'}
+);
+is($primary_ts, $standby_ts, "standby gives same value as primary");
+
+$primary->append_conf('postgresql.conf', 'track_commit_timestamp = off');
+$primary->restart;
+$primary->safe_psql('postgres', 'checkpoint');
+$primary_lsn = $primary->safe_psql('postgres', 'select pg_current_wal_lsn()');
+$standby->poll_query_until('postgres',
+ qq{SELECT '$primary_lsn'::pg_lsn <= pg_last_wal_replay_lsn()})
+ or die "standby never caught up";
+$standby->safe_psql('postgres', 'checkpoint');
+
+# This one should raise an error now
+my ($ret, $standby_ts_stdout, $standby_ts_stderr) = $standby->psql('postgres',
+ 'select ts.* from pg_class, pg_xact_commit_timestamp(xmin) ts where relname = \'t10\''
+);
+is($ret, 3, 'standby errors when primary turned feature off');
+is($standby_ts_stdout, '',
+ "standby gives no value when primary turned feature off");
+like(
+ $standby_ts_stderr,
+ qr/could not get commit timestamp data/,
+ 'expected error when primary turned feature off');
+
+done_testing();
diff --git a/src/test/modules/test_misc/t/099_ecpg_16678.pl b/src/test/modules/test_misc/t/099_ecpg_16678.pl
new file mode 100755
index 0000000000..53d1af7fe2
--- /dev/null
+++ b/src/test/modules/test_misc/t/099_ecpg_16678.pl
@@ -0,0 +1,56 @@
+# Run ecpg regression tests in a loop
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+use File::Basename;
+
+# Initialize primary node
+my $node_primary = PostgreSQL::Test::Cluster->new('p1');
+$node_primary->init(
+ 'auth_extra' => [ '--create-role', 'regress_ecpg_user1,regress_ecpg_user2' ]
+);
+
+$node_primary->start;
+
+my $outputdir = $PostgreSQL::Test::Utils::tmp_check;
+my $NUM_ITERATIONS = 2;
+
+for (my $i = 0; $i < $NUM_ITERATIONS; $i++)
+{
+my $extra_opts = $ENV{EXTRA_REGRESS_OPTS} || "";
+
+my $rc =
+ system("\"$outputdir/../../../../src/interfaces/ecpg/test/pg_regress_ecpg\" "
+ . " $extra_opts "
+ . "--dbname=ecpg1_regression,ecpg2_regression --create-role=regress_ecpg_user1,regress_ecpg_user2 "
+ . "--bindir= "
+ . "--host="
+ . $node_primary->host . " "
+ . "--port="
+ . $node_primary->port . " "
+ . "--outputdir=$outputdir "
+ . "--inputdir=$outputdir/../../../../src/interfaces/ecpg/test/ "
+ . "--expecteddir=../../../../src/interfaces/ecpg/test/ "
+ . ( "connect/test5 " x 50)
+);
+
+if ($rc != 0)
+{
+ # Dump out the regression diffs file, if there is one
+ my $diffs = "$outputdir/regression.diffs";
+ if (-e $diffs)
+ {
+ print "=== dumping $diffs ===\n";
+ print slurp_file($diffs);
+ print "=== EOF ===\n";
+ }
+}
+is($rc, 0, 'ecpg regression tests pass');
+($rc == 0) || last;
+}
+
+$node_primary->stop;
+
+done_testing();
diff --git a/src/test/modules/test_misc/t/099_pgbench_with_server_off.pl b/src/test/modules/test_misc/t/099_pgbench_with_server_off.pl
new file mode 100644
index 0000000000..409ef1621f
--- /dev/null
+++ b/src/test/modules/test_misc/t/099_pgbench_with_server_off.pl
@@ -0,0 +1,48 @@
+
+# Copyright (c) 2021-2023, PostgreSQL Global Development Group
+
+#
+# test pgbench with server stopped abruptly during the pgbench run
+#
+
+use strict;
+use warnings;
+
+use PostgreSQL::Test::Utils;
+use PostgreSQL::Test::Cluster;
+use Test::More;
+
+my $node = PostgreSQL::Test::Cluster->new('main');
+
+$node->init();
+$node->start;
+
+my $pb_timeout =
+ IPC::Run::timer($PostgreSQL::Test::Utils::timeout_default);
+my ($pb_stdin, $pb_stdout, $pb_stderr) = ('', '', '');
+
+$ENV{PGDATABASE} = 'postgres';
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+
+my $pb = IPC::Run::start(
+ [
+ 'pgbench',
+ '-i', '-s', '1000'
+ ],
+ '<',
+ \$pb_stdin,
+ '>',
+ \$pb_stdout,
+ '2>',
+ \$pb_stderr,
+ $pb_timeout);
+sleep(5);
+$node->stop('immediate');
+ok( pump_until(
+ $pb, $pb_timeout,
+ \$pb_stderr, qr/PQputline failed/),
+ 'pgbench reacted to server shutdown during copy');
+$pb->finish();
+
+done_testing();
diff --git a/src/test/ssl/meson.build b/src/test/ssl/meson.build
index abb30ab214..9228acf6fb 100644
--- a/src/test/ssl/meson.build
+++ b/src/test/ssl/meson.build
@@ -13,6 +13,7 @@ tests += {
't/001_ssltests.pl',
't/002_scram.pl',
't/003_sslinfo.pl',
+ 't/099_rare_ssl_failures.pl',
],
},
}
diff --git a/src/test/ssl/t/099_rare_ssl_failures-v12.pl b/src/test/ssl/t/099_rare_ssl_failures-v12.pl
new file mode 100644
index 0000000000..1c43f00301
--- /dev/null
+++ b/src/test/ssl/t/099_rare_ssl_failures-v12.pl
@@ -0,0 +1,84 @@
+use strict;
+use warnings;
+use PostgresNode;
+use TestLib;
+use Test::More tests => 100;
+
+use File::Copy;
+
+use FindBin;
+use lib $FindBin::RealBin;
+
+use SSLServer;
+
+#### Some configuration
+
+# This is the hostname used to connect to the server. This cannot be a
+# hostname, because the server certificate is always for the domain
+# postgresql-ssl-regression.test.
+my $SERVERHOSTADDR = '127.0.0.1';
+
+my $common_connstr;
+
+sub connect_fails
+{
+ my ($node, $connstr, $test_name, %params) = @_;
+
+ my $cmd = [
+ 'psql', '-X', '-A', '-t', '-c', "SELECT 'connected with $connstr'",
+ '-d', "$common_connstr $connstr" ];
+
+ my ($stdout, $stderr);
+
+ print("# Running: " . join(" ", @{$cmd}) . "\n");
+ my $result = IPC::Run::run $cmd, '>', \$stdout, '2>', \$stderr;
+
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: matches");
+ }
+}
+
+# The client's private key must not be world-readable, so take a copy
+# of the key stored in the code tree and update its permissions.
+copy("ssl/client.key", "ssl/client_tmp.key");
+chmod 0600, "ssl/client_tmp.key";
+
+#### Part 0. Set up the server.
+
+note "setting up data directory";
+my $node = get_new_node('master');
+$node->init;
+
+# PGHOST is enforced here to set up the node, subsequent connections
+# will use a dedicated connection string.
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+configure_test_server_for_ssl($node, $SERVERHOSTADDR, 'trust');
+switch_server_cert($node, 'server-cn-only');
+
+### Part 1. Run client-side tests.
+###
+### Test that libpq accepts/rejects the connection correctly, depending
+### on sslmode and whether the server's certificate looks correct. No
+### client certificate is used in these tests.
+
+note "running client tests";
+
+$common_connstr =
+"user=ssltestuser dbname=trustdb sslcert=invalid hostaddr=$SERVERHOSTADDR host=common-name.pg-ssltest.test";
+
+
+for (my $i = 1; $i <= 100; $i++) {
+print("iteration $i\n");
+connect_fails(
+ $node,
+ "user=ssltestuser sslcert=ssl/client-revoked.crt sslkey=ssl/client-revoked.key",
+ "certificate authorization fails with revoked client cert",
+ expected_stderr => qr/SSL error: sslv3 alert certificate revoked/,
+);
+}
+
+# clean up
+unlink "ssl/client_tmp.key";
diff --git a/src/test/ssl/t/099_rare_ssl_failures.pl b/src/test/ssl/t/099_rare_ssl_failures.pl
new file mode 100644
index 0000000000..0ca70c0ff6
--- /dev/null
+++ b/src/test/ssl/t/099_rare_ssl_failures.pl
@@ -0,0 +1,101 @@
+use strict;
+use warnings;
+use PostgreSQL::Test::Cluster;
+use PostgreSQL::Test::Utils;
+use Test::More;
+
+use File::Copy;
+
+use FindBin;
+use lib $FindBin::RealBin;
+
+use SSL::Server;
+
+#### Some configuration
+
+# This is the hostname used to connect to the server. This cannot be a
+# hostname, because the server certificate is always for the domain
+# postgresql-ssl-regression.test.
+my $SERVERHOSTADDR = '127.0.0.1';
+# This is the pattern to use in pg_hba.conf to match incoming connections.
+my $SERVERHOSTCIDR = '127.0.0.1/32';
+
+my $common_connstr;
+
+sub connect_fails
+{
+ my ($node, $connstr, $test_name, %params) = @_;
+
+ my $cmd = [
+ 'psql', '-X', '-A', '-t', '-c', "SELECT 'connected with $connstr'",
+ '-d', "$common_connstr $connstr" ];
+
+ my ($stdout, $stderr);
+
+ print("# Running: " . join(" ", @{$cmd}) . "\n");
+ my $result = IPC::Run::run $cmd, '>', \$stdout, '2>', \$stderr;
+
+ if (defined($params{expected_stderr}))
+ {
+ like($stderr, $params{expected_stderr}, "$test_name: matches");
+ }
+}
+
+my $ssl_server = SSL::Server->new();
+
+# The client's private key must not be world-readable, so take a copy
+# of the key stored in the code tree and update its permissions.
+copy("ssl/client.key", "ssl/client_tmp.key");
+chmod 0600, "ssl/client_tmp.key";
+
+#### Part 0. Set up the server.
+
+note "setting up data directory";
+my $node = PostgreSQL::Test::Cluster->new('primary');
+$node->init;
+
+# PGHOST is enforced here to set up the node, subsequent connections
+# will use a dedicated connection string.
+$ENV{PGHOST} = $node->host;
+$ENV{PGPORT} = $node->port;
+$node->start;
+
+$ssl_server->configure_test_server_for_ssl($node, $SERVERHOSTADDR,
+ $SERVERHOSTCIDR, 'trust');
+
+$ssl_server->switch_server_cert(
+ $node,
+ certfile => 'server-cn-only',
+ crldir => 'root+client-crldir');
+
+### Part 1. Run client-side tests.
+###
+### Test that libpq accepts/rejects the connection correctly, depending
+### on sslmode and whether the server's certificate looks correct. No
+### client certificate is used in these tests.
+
+note "running client tests";
+
+$common_connstr =
+"user=ssltestuser dbname=trustdb sslcert=invalid hostaddr=$SERVERHOSTADDR host=common-name.pg-ssltest.test";
+
+
+for (my $i = 1; $i <= 1000; $i++) {
+print("iteration $i\n");
+$node->connect_fails(
+ "$common_connstr user=ssltestuser sslcert=ssl/client-revoked.crt "
+ . $ssl_server->sslkey('client-revoked.key'),
+ "certificate authorization fails with revoked client cert with server-side CRL directory",
+ expected_stderr => qr/SSL error: sslv3 alert certificate revoked/,
+ # temporarily(?) skip this check due to timing issue
+ # log_like => [
+ # qr{Client certificate verification failed at depth 0: certificate revoked},
+ # qr{Failed certificate data \(unverified\): subject "/CN=ssltestuser", serial number 2315134995201656577, issuer "/CN=Test CA for PostgreSQL SSL regression test client certs"},
+ # ]
+);
+}
+
+# clean up
+unlink "ssl/client_tmp.key";
+
+done_testing();
: # git reset --hard b282fa88d~1 && git clean -dfx
: # git apply .../win-sock-tests-01.patch
: # cp .../v2-*.patch .
: # Setup an alias for python3 on Windows
: # cd .\\. && mklink c:\Python3\python3.exe c:\Python3\python.exe REM or
something like that
cd .\\. && @echo off
: # Multiply ssl tests
perl -i.bak -ne "print unless /ssl_/" "src/test/meson.build" && python3 -c "for
i in range(1,10+1):import os;import
shutil;sd=\"src/test/ssl/\";td=f\"src/test/ssl_{i}\";shutil.rmtree(td,ignore_errors=1);shutil.copytree(sd,
td);assert(os.system(f'perl -i.bak -pe \"s#( subdir.\'ssl\'.)#
subdir(\'ssl_{i}\'){chr(92)}n{chr(92)}1#\"
\"src/test/meson.build\"')==0);assert(os.system(f'perl -i.bak -pe
\"s#\'ssl\',#\'ssl_{i}\',#\" \"{td}/meson.build\"')==0)"
: # Multiply test_misc tests
perl -i.bak -ne "print unless /test_misc_/" "src/test/modules/meson.build" &&
python3 -c "for i in range(1,10+1):import os;import
shutil;pd=\"src/test/modules\";m=\"test_misc\";td=f\"{pd}/{m}_{i}\";shutil.rmtree(td,ignore_errors=1);shutil.copytree(f\"{pd}/{m}\",
td);assert(os.system(f'perl -i.bak -pe
\"s#(subdir.\'{m}\'.)#subdir(\'{m}_{i}\'){chr(92)}n{chr(92)}1#\"
\"{pd}/meson.build\"')==0);assert(os.system(f'perl -i.bak -pe \"s#(\'name\':
\'{m})#{chr(92)}1_{i}#\" \"{td}/meson.build\"')==0)"
: # Multiply postgres_fdw tests
perl -i.bak -ne "print unless /postgres_fdw_/" "contrib/meson.build" && python3
-c "for i in range(1,10+1):import os;import
shutil;pd=\"contrib\";m=\"postgres_fdw\";td=f\"{pd}/{m}_{i}\";shutil.rmtree(td,ignore_errors=1);shutil.copytree(f\"{pd}/{m}\",
td);assert(os.system(f'perl -i.bak -pe
\"s#(subdir.\'{m}\'.)#subdir(\'{m}_{i}\'){chr(92)}n{chr(92)}1#\"
\"{pd}/meson.build\"')==0);assert(os.system(f'perl -i.bak -pe \"s#(\'name\':
\'{m})#{chr(92)}1_{i}#\" \"{td}/meson.build\"')==0)"
cd .\\. && meson setup --wipe build -Dcassert=true -Db_pch=true
-Dextra_lib_dirs="C:\Program Files\OpenSSL-Win64\lib"
-Dextra_include_dirs="C:\Program Files\OpenSSL-Win64\include"
-DPG_TEST_EXTRA="ssl"
: && meson setup build -Dcassert=true -DPG_TEST_EXTRA="ssl"
cd build && ninja >ninja.log && meson test --suite setup
echo #
echo ========================================
echo Re: BUG 15598: PostgreSQL Error Code is not reported when connection
terminated due to idle-in-transaction timeout
meson test test_misc/099_case_15598
: # this test fails on Windows without 6051857fc due to "psql:<stdin>:6: server
closed the connection unexpectedly"
grep -r "psql stderr:" "testrun/test_misc/099_case_15598"
: # this test fails on Windows with 6051857fc due to "psql:<stdin>:6: could not
receive data from server: Software caused connection abort (0x00002745/10053)"
echo #
echo ========================================
echo Rare SSL failures on eelpout
python3 -c "NUMITERATIONS=10;NUMTESTS=10;import os;tsts='';exec('for i in
range(1,NUMTESTS+1): tsts+=f\"ssl_{i}/099_rare_ssl_failures \"'); exec('for i
in range(1,NUMITERATIONS+1):print(f\"iteration {i}\");
assert(os.system(f\"meson test --num-processes {NUMTESTS} {tsts}\") == 0)')"
: # this test fails on Windows sporadically with "server closed the connection
unexpectedly"
: # this test (v12-compatible variation) fails on 1f39a1c06~1 (on iterations 1,
1, 1 for me)
: # PG_TEST_EXTRA=ssl make -s check -C src/test/ssl; rm -rf src/test/ssl_*; for
i in `seq 20`; do cp -r src/test/ssl/ src/test/ssl_$i; sed
"s|src/test/ssl|src/test/ssl_$i|" -i src/test/ssl_$i/Makefile; done; for
((i=1;i<=10;i++)); do echo "ITERATION $i"; parallel --halt now,fail=1 -j20
--linebuffer --tag PG_TEST_EXTRA=ssl make -s check -C src/test/ssl_{}
PROVE_TESTS="t/099_rare_ssl_failures-v12.pl" NO_TEMP_INSTALL=1 ::: `seq 20` ||
break; done
echo #
echo ========================================
echo pgsql: Add tests for '-f' option in dropdb utility.
meson test test_misc/051_dropdb_force
: # this test fails on Windows without 6051857fc due to "psql:<stdin>:4: server
closed the connection unexpectedly"
grep -r "psql:<stdin>:" "testrun/test_misc/051_dropdb_force"
: # this test fails on Windows with 6051857fc due to "psql:<stdin>:4: could not
receive data from server: Software caused connection abort (0x00002745/10053)"
echo #
echo ========================================
echo libpq copy error handling busted
meson test test_misc/099_pgbench_with_server_off
: # this test fails before 7247e243a, to revert that commit:
: # git show 0245f8db3 src/interfaces/libpq/fe-secure.c | git apply -R && git
show a9e9a9f32 src/interfaces/libpq/fe-secure.c | git apply -R && git show
faa189c932d | git apply -R
: # git show 7247e243a | git apply -R
: # this test passes after 7247e243a, also survives 50 iterations x 10 parallel
processes
: # python3 -c "NUMITERATIONS=10;NUMTESTS=10;import os;tsts='';exec('for i in
range(1,NUMTESTS+1): tsts+=f\"test_misc_{i}/099_pgbench_with_server_off \"');
exec('for i in range(1,NUMITERATIONS+1):print(f\"iteration {i}\");
assert(os.system(f\"meson test --num-processes {NUMTESTS} {tsts}\") == 0)')"
echo #
echo ========================================
echo BUG 16678: The ecpg connect/test5 test sometimes fails on Windows
perl -i.bak -pe "s/(.*OK, try to read some data)/pg_usleep(10000L);\n\1/"
"../src/interfaces/libpq/fe-misc.c"
perl -i.bak -pe "s/(.*WalSndWait\(wakeEvents, sleeptime,
WAIT_EVENT_WAL_SENDER_MAIN\);)/\1\npg_usleep(11000L);/"
"../src/backend/replication/walsender.c"
: # sleep in walsender.c needed due to the anomaly "walsender cannot exit"
: # Without it, these tests fail on Linux and on Windows:
recovery/001_stream_rep recovery/009_twophase recovery/012_subtransactions
recovery/021_row_visibility recovery/032_relfilenode_reuse
recovery/035_standby_logical_decoding pg_rewind/008_min_recovery_point
ninja && meson test --suite setup
meson test ecpg/ecpg
meson test pg_dump/002_pg_dump
: # Failed tests, fixed with 6051857fc: pg_upgrade/002_pg_upgrade
isolation/isolation recovery/019_replslot_limit recovery/037_invalid_database
test_decoding/isolation ecpg/ecpg pg_amcheck/002_nonesuch pg_dump/002_pg_dump
pgbench/001_pgbench_with_server psql/001_basic scripts/011_clusterdb_all
scripts/091_reindexdb_all scripts/101_vacuumdb_all
cd .. && git show 6051857fc | git apply && cd build && ninja && meson test
--suite setup
meson test pg_upgrade/002_pg_upgrade isolation/isolation
recovery/019_replslot_limit recovery/037_invalid_database
test_decoding/isolation ecpg/ecpg pg_amcheck/002_nonesuch pg_dump/002_pg_dump
pgbench/001_pgbench_with_server psql/001_basic scripts/011_clusterdb_all
scripts/091_reindexdb_all scripts/101_vacuumdb_all
: # Test recovery/019_replslot_limit fails on Linux with the delay added, but
doesn't fail on Windows (with 6051857fc)
echo #
echo ========================================
echo MSVC SSL test failure
meson test ssl/001_ssltests
: # this test fails (on a first run, with the sleep added) after 6051857fc with
an error "server closed the connection unexpectedly"
cd .. && git show ed52c3707 | git apply && cd build && ninja && meson test
--suite setup
meson test ssl/001_ssltests
: # this test passes (with the sleep added) after 6051857fc + ed52c3707
echo #
echo ========================================
echo Why is src/test/modules/committs/t/002_standby.pl flaky?
cd ..
git restore "src/interfaces/libpq/fe-misc.c"
"src/backend/replication/walsender.c"
git show 6051857fc | git apply && git show ed52c3707 | git apply
cd build && ninja >ninja.log && meson test --suite setup
python3 -c "NUMITERATIONS=10;NUMTESTS=10;import os;tsts='';exec('for i in
range(1,NUMTESTS+1): tsts+=f\"test_misc_{i}/099_commit_ts_002_standby \"');
exec('for i in range(1,NUMITERATIONS+1):print(f\"iteration {i}\");
assert(os.system(f\"meson test --num-processes {NUMTESTS} {tsts}\") == 0)')"
: # this test failed on iterations 1, 3, 1 for me (without the sleep but with
6051857fc + ed52c3707)
python3 -c "NUMITERATIONS=10;NUMTESTS=1;import os;tsts='';exec('for i in
range(1,NUMTESTS+1): tsts+=f\"postgres_fdw_{i}/099_postgres_fdw_disconnect
\"'); exec('for i in range(1,NUMITERATIONS+1):print(f\"iteration {i}\");
assert(os.system(f\"meson test -t 0.2 --num-processes {NUMTESTS} {tsts}\") ==
0)')"
: # this test also failed (with TIMEOUT) on iterations 4, 6, 2 for me
cd .\\. && echo Killing postgres processes left behind failing tests &&
taskkill /F /FI "IMAGENAME eq postgres.exe"
cd .. && git show ed52c3707 | git apply -R && git show 6051857fc | git apply -R
&& cd build && ninja >ninja.log && meson test --suite setup
python3 -c "NUMITERATIONS=10;NUMTESTS=10;import os;tsts='';exec('for i in
range(1,NUMTESTS+1): tsts+=f\"test_misc_{i}/099_commit_ts_002_standby \"');
exec('for i in range(1,NUMITERATIONS+1):print(f\"iteration {i}\");
assert(os.system(f\"meson test --num-processes {NUMTESTS} {tsts}\") == 0)')"
: # 10 iterations succeeded (without the sleep and without 6051857fc +
ed52c3707)
echo #
echo ========================================
echo BUG 17391: While using --with-ssl=openssl and PG_TEST_EXTRA='ssl' options,
SSL tests fail on OpenBSD 7.0
: # Linux-only part; use make because meson.build doesn't play well with
libressl for now
: && cd ..
: # Revert faa189c93 (we also need non-essential changes a9e9a9f32 and
0245f8db3 above)
: && git show 0245f8db3 src/interfaces/libpq/fe-secure.c | git apply -R && git
show a9e9a9f32 src/interfaces/libpq/fe-secure.c | git apply -R && git show
faa189c93 | git apply -R
: && ( cd /tmp/ && rm -rf libressl*; wget
https://ftp.openbsd.org/pub/OpenBSD/LibreSSL/libressl-3.4.1.tar.gz && tar fax
libressl-3.4.1.tar.gz && cd libressl-3.4.1/ && ./configure -q
--prefix=/tmp/libressl && make -j`nproc` >/dev/null && make install >/dev/null )
: && PATH="/tmp/libressl/bin/:$PATH" sh -c "./configure --enable-tap-tests
--with-ssl=openssl --with-includes=/tmp/libressl/include
--with-libs=/tmp/libressl/lib && make -s -j8"
: && PG_TEST_EXTRA='ssl' LD_LIBRARY_PATH="/tmp/libressl/lib" make -s check -C
src/test/ssl PROVE_TESTS="t/001*"
: # this test fails without faa189c93
: && grep -r '# skip SSL_CERT_FILE is not supported'
src/test/ssl/tmp_check/log/ # make sure that LibreSSL is used
: && make -s clean
: # Undo all the changes
: && git show faa189c93 | git apply && git show a9e9a9f32
src/interfaces/libpq/fe-secure.c | git apply && git show 0245f8db3
src/interfaces/libpq/fe-secure.c | git apply
: && PATH="/tmp/libressl/bin/:$PATH" sh -c "./configure --enable-tap-tests
--with-ssl=openssl --with-includes=/tmp/libressl/include
--with-libs=/tmp/libressl/lib && make -s -j8"
: && PG_TEST_EXTRA='ssl' LD_LIBRARY_PATH="/tmp/libressl/lib" make -s check -C
src/test/ssl PROVE_TESTS="t/001*"
: # this test passes with faa189c93 (on current master)
: && grep -r '# skip SSL_CERT_FILE is not supported'
src/test/ssl/tmp_check/log/ # make sure that LibreSSL is used
: && make -s clean && make -s distclean && cd build
echo #
echo ========================================
echo Applying all the latest patches
cd ..
git apply v2-0001-simplehash-Allow-raw-memory-to-be-freed.patch
v2-0002-simplehash-Allow-raw-allocation-to-fail.patch
git apply v2-0003-Redesign-Windows-socket-event-management.patch
git apply v2-0004-Remove-pgwin32_select.patch
git apply v2-0005-Refactor-pgwin32_waitforsinglesocket-to-share-eve.patch
git apply v2-0006-Reinstate-graceful-shutdown-changes-for-Windows.patch
cd build && ninja >ninja.log && meson test --suite setup
echo #
echo ========================================
echo Checking whether commit_ts_002_standby is still flaky...
python3 -c "NUMITERATIONS=10;NUMTESTS=10;import os;tsts='';exec('for i in
range(1,NUMTESTS+1): tsts+=f\"test_misc_{i}/099_commit_ts_002_standby \"');
exec('for i in range(1,NUMITERATIONS+1):print(f\"iteration {i}\");
assert(os.system(f\"meson test --num-processes {NUMTESTS} {tsts}\") == 0)')"
: # no failures on Linux and Windows
echo Checking whether 099_postgres_fdw_disconnect is still flaky...
python3 -c "NUMITERATIONS=10;NUMTESTS=1;import os;tsts='';exec('for i in
range(1,NUMTESTS+1): tsts+=f\"postgres_fdw_{i}/099_postgres_fdw_disconnect
\"'); exec('for i in range(1,NUMITERATIONS+1):print(f\"iteration {i}\");
assert(os.system(f\"meson test -t 0.2 --num-processes {NUMTESTS} {tsts}\") ==
0)')"
: # no failures on Linux and Windows
: # tests 051_dropdb_force, 099_case_15598 still fail on Windows due to the
anomaly "send() before recv() invalidates a closed socket"
