I started looking at the "Improve compactify_tuples and
PageRepairFragmentation" patch, and set up a little performance test of
WAL replay. I ran pgbench, scale 5, to generate about 1 GB of WAL, and
timed how long it takes to replay that WAL. To focus purely on CPU
overhead, I kept the data directory in /dev/shm/.
Profiling that, without any patches applied, I noticed that a lot of
time was spent in read()s on the postmaster-death pipe, i.e. in
PostmasterIsAlive(). We call that between *every* WAL record.
As a quick test to see how much that matters, I commented out the
PostmasterIsAlive() call from HandleStartupProcInterrupts(). On
unpatched master, replaying that 1 GB of WAL takes about 20 seconds on
my laptop. Without the PostmasterIsAlive() call, 17 seconds.
That seems like an utter waste of time. I'm almost inclined to call that
a performance bug. As a straightforward fix, I'd suggest that we call
HandleStartupProcInterrupts() in the WAL redo loop, not on every record,
but only e.g. every 32 records. That would make the main redo loop less
responsive to shutdown, SIGHUP, or postmaster death, but that seems OK.
There are also calls to HandleStartupProcInterrupts() in the various
other loops, that wait for new WAL to arrive or recovery delay, so this
would only affect the case where we're actively replaying records.
- Heikki
>From 596d3804c784b06f2125aa7727b82a265b08ccfb Mon Sep 17 00:00:00 2001
From: Heikki Linnakangas <heikki.linnakan...@iki.fi>
Date: Thu, 5 Apr 2018 10:18:01 +0300
Subject: [PATCH 1/1] Call HandleStartupProcInterrupts() less frequently in WAL
redo.
HandleStartupProcInterrupts() calls PostmasterIsAlive(), which calls read()
on the postmaster death watch pipe. That's relatively expensive, when we do
it between every WAL record.
---
src/backend/access/transam/xlog.c | 11 +++++++++--
1 file changed, 9 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index b4fd8395b7..3406c58c9b 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -7072,6 +7072,7 @@ StartupXLOG(void)
{
ErrorContextCallback errcallback;
TimestampTz xtime;
+ uint32 records_replayed = 0;
InRedo = true;
@@ -7105,8 +7106,13 @@ StartupXLOG(void)
}
#endif
- /* Handle interrupt signals of startup process */
- HandleStartupProcInterrupts();
+ /*
+ * Handle interrupt signals of startup process. This includes
+ * a call to PostmasterIsAlive(), which isn't totally free, so
+ * don't do this on every record, to avoid the overhead.
+ */
+ if (records_replayed % 32 == 0)
+ HandleStartupProcInterrupts();
/*
* Pause WAL replay, if requested by a hot-standby session via
@@ -7269,6 +7275,7 @@ StartupXLOG(void)
/* Remember this record as the last-applied one */
LastRec = ReadRecPtr;
+ records_replayed++;
/* Allow read-only connections if we're consistent now */
CheckRecoveryConsistency();
--
2.11.0
