On 2020-11-20 16:47, Sergei Kornilov wrote:
Hmm... Good question. How about putting CheckForStandbyTrigger() in a wait loop, but
reporting FATAL with an appropriate message, such as "promotion is not possible
because of insufficient parameter settings"?
Also it suits me if we only document that we ignore promote here. I don't think
this is an important case. And yes, it's not easy to allow promotion, since we
have already updated control file.
Probably we need pause only after we reached consistency?
Here is an updated patch that implements both of these points. I have
used hot standby active instead of reached consistency. I guess
arguments could be made either way, but the original use case really
cared about hot standby.
--
Peter Eisentraut
2ndQuadrant, an EDB company
https://www.2ndquadrant.com/
From 2d93c2918af7ff186aa4a6a2327286b6eb1a2859 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut <pe...@eisentraut.org>
Date: Mon, 30 Nov 2020 19:21:40 +0100
Subject: [PATCH v6] Pause recovery for insufficient parameter settings
When certain parameters are changed on a physical replication primary,
this is communicated to standbys using the XLOG_PARAMETER_CHANGE WAL
record. The standby then checks whether its own settings are at least
as big as the ones on the primary. If not, the standby shuts down
with a fatal error.
This patch changes this behavior for hot standbys to pause recovery at
that point instead. That allows read traffic on the standby to
continue while database administrators figure out next steps. When
recovery is unpaused, the server shuts down (as before). The idea is
to fix the parameters while recovery is paused and then restart when
there is a maintenance window.
Discussion:
https://www.postgresql.org/message-id/flat/4ad69a4c-cc9b-0dfe-0352-8b1b0cd36...@2ndquadrant.com
---
doc/src/sgml/high-availability.sgml | 51 +++++++++++++++++++------
src/backend/access/transam/xlog.c | 59 ++++++++++++++++++++++++++---
2 files changed, 93 insertions(+), 17 deletions(-)
diff --git a/doc/src/sgml/high-availability.sgml
b/doc/src/sgml/high-availability.sgml
index 19d7bd2b28..71f7f58cb0 100644
--- a/doc/src/sgml/high-availability.sgml
+++ b/doc/src/sgml/high-availability.sgml
@@ -2120,18 +2120,14 @@ <title>Administrator's Overview</title>
</para>
<para>
- The setting of some parameters on the standby will need reconfiguration
- if they have been changed on the primary. For these parameters,
- the value on the standby must
- be equal to or greater than the value on the primary.
- Therefore, if you want to increase these values, you should do so on all
- standby servers first, before applying the changes to the primary server.
- Conversely, if you want to decrease these values, you should do so on the
- primary server first, before applying the changes to all standby servers.
- If these parameters
- are not set high enough then the standby will refuse to start.
- Higher values can then be supplied and the server
- restarted to begin recovery again. These parameters are:
+ The settings of some parameters determine the size of shared memory for
+ tracking transaction IDs, locks, and prepared transactions. These shared
+ memory structures must be no smaller on a standby than on the primary in
+ order to ensure that the standby does not run out of shared memory during
+ recovery. For example, if the primary had used a prepared transaction but
+ the standby had not allocated any shared memory for tracking prepared
+ transactions, then recovery could not continue until the standby's
+ configuration is changed. The parameters affected are:
<itemizedlist>
<listitem>
@@ -2160,6 +2156,37 @@ <title>Administrator's Overview</title>
</para>
</listitem>
</itemizedlist>
+
+ The easiest way to ensure this does not become a problem is to have these
+ parameters set on the standbys to values equal to or greater than on the
+ primary. Therefore, if you want to increase these values, you should do
+ so on all standby servers first, before applying the changes to the
+ primary server. Conversely, if you want to decrease these values, you
+ should do so on the primary server first, before applying the changes to
+ all standby servers. Keep in mind that when a standby is promoted, it
+ becomes the new reference for the required parameter settings for the
+ standbys that follow it. Therefore, to avoid this becoming a problem
+ during a switchover or failover, it is recommended to keep these settings
+ the same on all standby servers.
+ </para>
+
+ <para>
+ The WAL tracks changes to these parameters on the
+ primary. If a hot standby processes WAL that indicates that the current
+ value on the primary is higher than its own value, it will log a warning
+ and pause recovery, for example:
+<screen>
+WARNING: hot standby is not possible because of insufficient parameter
settings
+DETAIL: max_connections = 80 is a lower setting than on the primary server,
where its value was 100.
+LOG: recovery has paused
+DETAIL: If recovery is unpaused, the server will shut down.
+HINT: You can then restart the server after making the necessary
configuration changes.
+</screen>
+ At that point, the settings on the standby need to be updated and the
+ instance restarted before recovery can continue. If the standby is not a
+ hot standby, then when it encounters the incompatible parameter change, it
+ will shut down immediately without pausing, since there is then no value
+ in keeping it up.
</para>
<para>
diff --git a/src/backend/access/transam/xlog.c
b/src/backend/access/transam/xlog.c
index 13f1d8c3dc..f9a41e0dfb 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6232,12 +6232,61 @@ static void
RecoveryRequiresIntParameter(const char *param_name, int currValue, int
minValue)
{
if (currValue < minValue)
- ereport(ERROR,
+ {
+ if (LocalHotStandbyActive)
+ {
+ bool warned_for_promote = false;
+
+ ereport(WARNING,
+
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("hot standby is not possible
because of insufficient parameter settings"),
+ errdetail("%s = %d is a lower setting
than on the primary server, where its value was %d.",
+ param_name,
+ currValue,
+ minValue)));
+
+ SetRecoveryPause(true);
+
+ ereport(LOG,
+ (errmsg("recovery has paused"),
+ errdetail("If recovery is unpaused,
the server will shut down."),
+ errhint("You can then restart the
server after making the necessary configuration changes.")));
+
+ while (RecoveryIsPaused())
+ {
+ HandleStartupProcInterrupts();
+
+ if (CheckForStandbyTrigger())
+ {
+ if (!warned_for_promote)
+ ereport(WARNING,
+
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+
errmsg("promotion is not possible because of insufficient parameter settings"),
+ /* Repeat the
detail from above so it's easy to find in the log. */
+ errdetail("%s
= %d is a lower setting than on the primary server, where its value was %d.",
+
param_name,
+
currValue,
+
minValue),
+
errhint("Restart the server after making the necessary configuration
changes.")));
+ warned_for_promote = true;
+ }
+
+
pgstat_report_wait_start(WAIT_EVENT_RECOVERY_PAUSE);
+ pg_usleep(1000000L); /* 1000 ms */
+ pgstat_report_wait_end();
+ }
+ }
+
+ ereport(FATAL,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("hot standby is not possible because %s
= %d is a lower setting than on the primary server (its value was %d)",
- param_name,
- currValue,
- minValue)));
+ errmsg("recovery aborted because of
insufficient parameter settings"),
+ /* Repeat the detail from above so it's easy
to find in the log. */
+ errdetail("%s = %d is a lower setting than on
the primary server, where its value was %d.",
+ param_name,
+ currValue,
+ minValue),
+ errhint("You can restart the server after
making the necessary configuration changes.")));
+ }
}
/*
--
2.29.2
