On Tue, Dec 7, 2021 at 12:58 AM Bossart, Nathan <bossa...@amazon.com> wrote:
>
> On 12/6/21, 4:34 AM, "Bharath Rupireddy"
> <bharath.rupireddyforpostg...@gmail.com> wrote:
> > While the database is performing end-of-recovery checkpoint, the
> > control file gets updated with db state as "shutting down" in
> > CreateCheckPoint (see the code snippet at [1]) and at the end it sets
> > it back to "shut down" for a brief moment and then finally to "in
> > production". If the end-of-recovery checkpoint takes a lot of time or
> > the db goes down during the end-of-recovery checkpoint for whatever
> > reasons, the control file ends up having the wrong db state.
> >
> > Should we add a new db state something like
> > DB_IN_END_OF_RECOVERY_CHECKPOINT/"in end-of-recovery checkpoint" or
> > something else to represent the correct state?
>
> This seems like a reasonable change to me. From a quick glance, it
> looks like it should be a simple fix that wouldn't add too much
> divergence between the shutdown and end-of-recovery checkpoint code
> paths.
Here's a patch that I've come up with. Please see if this looks okay
and let me know if we want to take it forward so that I can add a CF
entry.
The new status one would see is as follows:
bharath@bharathubuntu3:~/postgres/inst/bin$ ./pg_controldata -D data
pg_control version number: 1500
Catalog version number: 202111301
Database system identifier: 7038867865889221935
Database cluster state: in end-of-recovery checkpoint
pg_control last modified: Tue Dec 7 08:06:18 2021
Latest checkpoint location: 0/14D24A0
Latest checkpoint's REDO location: 0/14D24A0
Latest checkpoint's REDO WAL file: 000000010000000000000001
Regards,
Bharath Rupireddy.
From 8cc1dfec87a53d3b423712380c934243e93009ac Mon Sep 17 00:00:00 2001
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: Tue, 7 Dec 2021 07:53:53 +0000
Subject: [PATCH v1] Add DB_IN_END_OF_RECOVERY_CHECKPOINT state for control
file
While the database is performing end-of-recovery checkpoint, the
control file gets updated with db state as "shutting down" in
CreateCheckPoint and at the end it sets it back to "shut down" for
a brief moment and then finally to "in production". If the
end-of-recovery checkpoint takes a lot of time or the db goes down
during the end-of-recovery checkpoint for whatever reasons, the
control file ends up having the wrong db state.
This patch adds a new db state to control file, called
DB_IN_END_OF_RECOVERY_CHECKPOINT or "in end-of-recovery checkpoint"
to represent the correct state of the database server.
---
src/backend/access/transam/xlog.c | 13 +++++++++++--
src/bin/pg_controldata/pg_controldata.c | 2 ++
src/include/catalog/pg_control.h | 3 ++-
3 files changed, 15 insertions(+), 3 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index d894af310a..a31a8dda35 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -6747,6 +6747,12 @@ StartupXLOG(void)
" and you might need to choose an earlier recovery target.")));
break;
+ case DB_IN_END_OF_RECOVERY_CHECKPOINT:
+ ereport(LOG,
+ (errmsg("database system was interrupted while in end-of-recovery checkpoint at %s",
+ str_time(ControlFile->time))));
+ break;
+
case DB_IN_PRODUCTION:
ereport(LOG,
(errmsg("database system was interrupted; last known up at %s",
@@ -9140,7 +9146,10 @@ CreateCheckPoint(int flags)
if (shutdown)
{
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- ControlFile->state = DB_SHUTDOWNING;
+ if (flags & CHECKPOINT_END_OF_RECOVERY)
+ ControlFile->state = DB_IN_END_OF_RECOVERY_CHECKPOINT;
+ else
+ ControlFile->state = DB_SHUTDOWNING;
UpdateControlFile();
LWLockRelease(ControlFileLock);
}
@@ -9406,7 +9415,7 @@ CreateCheckPoint(int flags)
* Update the control file.
*/
LWLockAcquire(ControlFileLock, LW_EXCLUSIVE);
- if (shutdown)
+ if (flags & CHECKPOINT_IS_SHUTDOWN)
ControlFile->state = DB_SHUTDOWNED;
ControlFile->checkPoint = ProcLastRecPtr;
ControlFile->checkPointCopy = checkPoint;
diff --git a/src/bin/pg_controldata/pg_controldata.c b/src/bin/pg_controldata/pg_controldata.c
index f911f98d94..e47b90143e 100644
--- a/src/bin/pg_controldata/pg_controldata.c
+++ b/src/bin/pg_controldata/pg_controldata.c
@@ -63,6 +63,8 @@ dbState(DBState state)
return _("in crash recovery");
case DB_IN_ARCHIVE_RECOVERY:
return _("in archive recovery");
+ case DB_IN_END_OF_RECOVERY_CHECKPOINT:
+ return _("in end-of-recovery checkpoint");
case DB_IN_PRODUCTION:
return _("in production");
}
diff --git a/src/include/catalog/pg_control.h b/src/include/catalog/pg_control.h
index 749bce0cc6..7ea0fdb273 100644
--- a/src/include/catalog/pg_control.h
+++ b/src/include/catalog/pg_control.h
@@ -22,7 +22,7 @@
/* Version identifier for this pg_control format */
-#define PG_CONTROL_VERSION 1300
+#define PG_CONTROL_VERSION 1500
/* Nonce key length, see below */
#define MOCK_AUTH_NONCE_LEN 32
@@ -92,6 +92,7 @@ typedef enum DBState
DB_SHUTDOWNING,
DB_IN_CRASH_RECOVERY,
DB_IN_ARCHIVE_RECOVERY,
+ DB_IN_END_OF_RECOVERY_CHECKPOINT,
DB_IN_PRODUCTION
} DBState;
--
2.25.1
