On Mon, Apr 13, 2020 at 5:14 PM Andres Freund <and...@anarazel.de> wrote:
> FWIW, I think the part that is currently harder to fix is the time->xmin
> mapping and some related pieces. Second comes the test
> infrastructure. Compared to those, adding additional checks for old
> snapshots wouldn't be too hard - although I'd argue that the approach of
> sprinkling these tests everywhere isn't that scalable...
Just trying out some ideas here... I suppose the wrapping problem
just requires something along the lines of the attached, but now I'm
wondering how to write decent tests for it. Using the
pg_clobber_current_snapshot_timestamp() function I mentioned in
Robert's time->xmin thread, it's easy to build up a time map without
resorting to sleeping etc, with something like:
select pg_clobber_current_snapshot_timestamp('3000-01-01 00:00:00Z');
select pg_current_xact_id();
select pg_clobber_current_snapshot_timestamp('3000-01-01 00:01:00Z');
select pg_current_xact_id();
select pg_clobber_current_snapshot_timestamp('3000-01-01 00:02:00Z');
select pg_current_xact_id();
select pg_clobber_current_snapshot_timestamp('3000-01-01 00:03:00Z');
select pg_current_xact_id();
select pg_clobber_current_snapshot_timestamp('3000-01-01 00:04:00Z');
Then of course frozenXID can be advanced with eg update pg_database
set datallowconn = 't' where datname = 'template0', then vacuumdb
--freeze --all, and checked before and after with Robert's
pg_old_snapshot_time_mapping() SRF to see that it's truncated. But
it's not really the level of stuff we'd ideally mess with in
pg_regress tests and I don't see any precent, so I guess maybe I'll
need to go and figure out how to write some perl.
From 69d0f7d843a8145fc8cdbd6a56b948ee04d486b9 Mon Sep 17 00:00:00 2001
From: Thomas Munro <thomas.mu...@gmail.com>
Date: Fri, 17 Apr 2020 15:18:49 +1200
Subject: [PATCH] Truncate old snapshot XIDs before truncating CLOG.
---
src/backend/commands/vacuum.c | 3 +++
src/backend/utils/time/snapmgr.c | 21 +++++++++++++++++++++
src/include/utils/snapmgr.h | 1 +
3 files changed, 25 insertions(+)
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 5a110edb07..37ead45fa5 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -1627,6 +1627,9 @@ vac_truncate_clog(TransactionId frozenXID,
*/
AdvanceOldestCommitTsXid(frozenXID);
+ /* Make sure snapshot_too_old drops old XIDs. */
+ TruncateOldSnapshotTimeMapping(frozenXID);
+
/*
* Truncate CLOG, multixact and CommitTs to the oldest computed value.
*/
diff --git a/src/backend/utils/time/snapmgr.c b/src/backend/utils/time/snapmgr.c
index 72b2c61a07..d604e69270 100644
--- a/src/backend/utils/time/snapmgr.c
+++ b/src/backend/utils/time/snapmgr.c
@@ -1998,6 +1998,27 @@ MaintainOldSnapshotTimeMapping(TimestampTz whenTaken, TransactionId xmin)
}
+/*
+ * Remove old xids from the timing map, so the CLOG can be truncated.
+ */
+void
+TruncateOldSnapshotTimeMapping(TransactionId frozenXID)
+{
+ LWLockAcquire(OldSnapshotTimeMapLock, LW_EXCLUSIVE);
+ while (oldSnapshotControl->count_used > 0 &&
+ TransactionIdPrecedes(oldSnapshotControl->xid_by_minute[oldSnapshotControl->head_offset],
+ frozenXID))
+ {
+ oldSnapshotControl->head_timestamp += USECS_PER_MINUTE;
+ oldSnapshotControl->head_offset =
+ (oldSnapshotControl->head_offset + 1) %
+ OLD_SNAPSHOT_TIME_MAP_ENTRIES;
+ oldSnapshotControl->count_used--;
+ }
+ LWLockRelease(OldSnapshotTimeMapLock);
+}
+
+
/*
* Setup a snapshot that replaces normal catalog snapshots that allows catalog
* access to behave just like it did at a certain point in the past.
diff --git a/src/include/utils/snapmgr.h b/src/include/utils/snapmgr.h
index b28d13ce84..4f53aad956 100644
--- a/src/include/utils/snapmgr.h
+++ b/src/include/utils/snapmgr.h
@@ -135,6 +135,7 @@ extern TransactionId TransactionIdLimitedForOldSnapshots(TransactionId recentXmi
Relation relation);
extern void MaintainOldSnapshotTimeMapping(TimestampTz whenTaken,
TransactionId xmin);
+extern void TruncateOldSnapshotTimeMapping(TransactionId frozenXID);
extern char *ExportSnapshot(Snapshot snapshot);
--
2.20.1
