On 30.05.2024 10:33, Alena Rybakina wrote:
I suggest gathering information about vacuum resource consumption for
processing indexes and tables and storing it in the table and index
relationships (for example, PgStat_StatTabEntry structure like it has
realized for usual statistics). It will allow us to determine how well
the vacuum is configured and evaluate the effect of overhead on the
system at the strategic level, the vacuum has gathered this
information already, but this valuable information doesn't store it.
My colleagues and I have prepared a patch that can help to solve this
problem.
We are open to feedback.
--
Regards,
Alena Rybakina
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
From dfda656b35be2a73c076cb723fdeb917630e61e3 Mon Sep 17 00:00:00 2001
From: Alena Rybakina <a.rybak...@postgrespro.ru>
Date: Thu, 30 May 2024 11:02:10 -0700
Subject: [PATCH] Machinery for grabbing an extended vacuum statistics on
relations. Remember, statistic on heap and index relations a bit different.
Value of total_blks_hit, total_blks_read, total_blks_dirtied are number of
hitted, missed and dirtied pages in shared buffers during a vacuum operation
respectively.
total_blks_dirtied means 'dirtied only by this action'. So, if this page was
dirty before the vacuum operation, it doesn't count this page as 'dirtied'.
The tuples_deleted parameter is the number of tuples cleaned up by the vacuum
operation.
The delay_time value means total vacuum sleep time in vacuum delay point.
The pages_removed value is the number of pages by which the physical data
storage of the relation was reduced.
The value of pages_deleted parameter is the number of freed pages in the table
(file size may not have changed).
Interruptions number of (auto)vacuum process during vacuuming of a relation.
We report from the vacuum_error_callback routine. So we can log all ERROR
reports. In the case of autovacuum we can report SIGINT signals too.
It maybe dangerous to make such complex task (send) in an error callback -
we can catch ERROR in ERROR problem. But it looks like we have so small
chance to stuck into this problem. So, let's try to use.
This parameter relates to a problem, covered by b19e4250.
Tracking of IO during an (auto)vacuum operation.
Introduced variables blk_read_time and blk_write_time tracks only access to
buffer pages and flushing them to disk. Reading operation is trivial, but
writing measurement technique is not obvious.
So, during a vacuum writing time can be zero incremented because no any flushing
operations were performed.
System time and user time are parameters that describes how much time a vacuum
operation has spent in executing of code in user space and kernel space
accordingly. Also, accumulate total time of a vacuum that is a diff between
timestamps in start and finish points in the vacuum code.
Remember about idle time, when vacuum waited for IO and locks, so total time
isn't equal a sum of user and system time, but no less.
pages_frozen - number of pages that are marked as frozen in vm during vacuum.
This parameter is incremented if page is marked as all-frozen.
pages_all_visible - number of pages that are marked as all-visible in vm during
vacuum.
Authors: Alena Rybakina <lena.riback...@yandex.ru>, Andrei Lepikhov <a.lepik...@postgrespro.ru>, Andrei Zubkov <a.zub...@postgrespro.ru>
---
src/backend/access/heap/vacuumlazy.c | 245 +++++++++-
src/backend/access/heap/visibilitymap.c | 13 +
src/backend/catalog/system_views.sql | 123 +++++
src/backend/commands/vacuum.c | 4 +
src/backend/commands/vacuumparallel.c | 1 +
src/backend/utils/activity/pgstat.c | 117 +++--
src/backend/utils/activity/pgstat_database.c | 1 +
src/backend/utils/activity/pgstat_relation.c | 52 ++-
src/backend/utils/adt/pgstatfuncs.c | 290 ++++++++++++
src/backend/utils/error/elog.c | 13 +
src/include/catalog/pg_proc.dat | 28 +-
src/include/commands/vacuum.h | 1 +
src/include/pgstat.h | 118 ++++-
src/include/utils/elog.h | 2 +-
src/include/utils/pgstat_internal.h | 36 +-
.../expected/vacuum-extended-statistic.out | 419 ++++++++++++++++++
.../isolation/expected/vacuum-extending.out | 68 +++
src/test/isolation/isolation_schedule | 2 +
.../specs/vacuum-extended-statistic.spec | 179 ++++++++
.../isolation/specs/vacuum-extending.spec | 58 +++
src/test/regress/expected/opr_sanity.out | 9 +-
src/test/regress/expected/rules.out | 79 ++++
22 files changed, 1812 insertions(+), 46 deletions(-)
create mode 100644 src/test/isolation/expected/vacuum-extended-statistic.out
create mode 100644 src/test/isolation/expected/vacuum-extending.out
create mode 100644 src/test/isolation/specs/vacuum-extended-statistic.spec
create mode 100644 src/test/isolation/specs/vacuum-extending.spec
diff --git a/src/backend/access/heap/vacuumlazy.c b/src/backend/access/heap/vacuumlazy.c
index 8145ea8fc3f..aa84f30443f 100644
--- a/src/backend/access/heap/vacuumlazy.c
+++ b/src/backend/access/heap/vacuumlazy.c
@@ -168,6 +168,9 @@ typedef struct LVRelState
char *dbname;
char *relnamespace;
char *relname;
+ Oid reloid;
+ Oid indoid;
+
char *indname; /* Current index name */
BlockNumber blkno; /* used only for heap operations */
OffsetNumber offnum; /* used only for heap operations */
@@ -194,6 +197,8 @@ typedef struct LVRelState
BlockNumber lpdead_item_pages; /* # pages with LP_DEAD items */
BlockNumber missed_dead_pages; /* # pages with missed dead tuples */
BlockNumber nonempty_pages; /* actually, last nonempty page + 1 */
+ BlockNumber set_frozen_pages; /* pages are marked as frozen in vm during vacuum */
+ BlockNumber set_all_visible_pages; /* pages are marked as all-visible in vm during vacuum */
/* Statistics output by us, for table */
double new_rel_tuples; /* new estimated total # of tuples */
@@ -226,6 +231,32 @@ typedef struct LVSavedErrInfo
VacErrPhase phase;
} LVSavedErrInfo;
+/*
+ * Cut-off values of parameters which changes implicitly during a vacuum
+ * process.
+ * Vacuum can't control their values, so we should store them before and after
+ * the processing.
+ */
+typedef struct LVExtStatCounters
+{
+ TimestampTz time;
+ PGRUsage ru;
+ WalUsage walusage;
+ BufferUsage bufusage;
+ int64 VacuumPageMiss;
+ int64 VacuumPageHit;
+ int64 VacuumPageDirty;
+ double VacuumDelayTime;
+ PgStat_Counter blocks_fetched;
+ PgStat_Counter blocks_hit;
+} LVExtStatCounters;
+
+typedef struct LVExtStatCountersIdx
+{
+ LVExtStatCounters common;
+ int64 pages_deleted;
+ int64 tuples_removed;
+} LVExtStatCountersIdx;
/* non-export function prototypes */
static void lazy_scan_heap(LVRelState *vacrel);
@@ -279,6 +310,158 @@ static void update_vacuum_error_info(LVRelState *vacrel,
static void restore_vacuum_error_info(LVRelState *vacrel,
const LVSavedErrInfo *saved_vacrel);
+/* ----------
+ * extvac_stats_start() -
+ *
+ * Save cut-off values of extended vacuum counters before start of a relation
+ * processing.
+ * ----------
+ */
+static void
+extvac_stats_start(Relation rel, LVExtStatCounters *counters)
+{
+ TimestampTz starttime;
+ PGRUsage ru0;
+
+ memset(counters, 0, sizeof(LVExtStatCounters));
+
+ pg_rusage_init(&ru0);
+ starttime = GetCurrentTimestamp();
+
+ counters->ru = ru0;
+ counters->time = starttime;
+ counters->walusage = pgWalUsage;
+ counters->bufusage = pgBufferUsage;
+ counters->VacuumPageMiss = VacuumPageMiss;
+ counters->VacuumPageHit = VacuumPageHit;
+ counters->VacuumPageDirty = VacuumPageDirty;
+ counters->VacuumDelayTime = VacuumDelayTime;
+ counters->blocks_fetched = 0;
+ counters->blocks_hit = 0;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ counters->blocks_fetched = rel->pgstat_info->counts.blocks_fetched;
+ counters->blocks_hit = rel->pgstat_info->counts.blocks_hit;
+}
+
+/* ----------
+ * extvac_stats_end() -
+ *
+ * Called to finish an extended vacuum statistic gathering and form a report.
+ * ----------
+ */
+static void
+extvac_stats_end(Relation rel, LVExtStatCounters *counters,
+ ExtVacReport *report)
+{
+ WalUsage walusage;
+ BufferUsage bufusage;
+ TimestampTz endtime;
+ long secs;
+ int usecs;
+ PGRUsage ru1;
+
+ /* Calculate diffs of global stat parameters on WAL and buffer usage. */
+ memset(&walusage, 0, sizeof(WalUsage));
+ WalUsageAccumDiff(&walusage, &pgWalUsage, &counters->walusage);
+
+ memset(&bufusage, 0, sizeof(BufferUsage));
+ BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &counters->bufusage);
+
+ endtime = GetCurrentTimestamp();
+ TimestampDifference(counters->time, endtime, &secs, &usecs);
+
+ memset(report, 0, sizeof(ExtVacReport));
+
+ /*
+ * Fill additional statistics on a vacuum processing operation.
+ */
+ report->total_blks_read = VacuumPageMiss - counters->VacuumPageMiss;
+ report->total_blks_hit = VacuumPageHit - counters->VacuumPageHit;
+ report->total_blks_dirtied = VacuumPageDirty - counters->VacuumPageDirty;
+ report->total_blks_written = bufusage.shared_blks_written;
+
+ report->wal_records = walusage.wal_records;
+ report->wal_fpi = walusage.wal_fpi;
+ report->wal_bytes = walusage.wal_bytes;
+
+ report->blk_read_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_read_time);
+ report->blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_read_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.local_blk_write_time);
+ report->blk_write_time = INSTR_TIME_GET_MILLISEC(bufusage.shared_blk_write_time);
+ report->delay_time = VacuumDelayTime - counters->VacuumDelayTime;
+
+ /*
+ * Get difference of a system time and user time values in milliseconds.
+ * Use floating point representation to show tails of time diffs.
+ */
+ pg_rusage_init(&ru1);
+ report->system_time =
+ (ru1.ru.ru_stime.tv_sec - counters->ru.ru.ru_stime.tv_sec) * 1000. +
+ (ru1.ru.ru_stime.tv_usec - counters->ru.ru.ru_stime.tv_usec) * 0.001;
+ report->user_time =
+ (ru1.ru.ru_utime.tv_sec - counters->ru.ru.ru_utime.tv_sec) * 1000. +
+ (ru1.ru.ru_utime.tv_usec - counters->ru.ru.ru_utime.tv_usec) * 0.001;
+ report->total_time = secs * 1000. + usecs / 1000.;
+
+ if (!rel->pgstat_info || !pgstat_track_counts)
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+ return;
+
+ report->blks_fetched =
+ rel->pgstat_info->counts.blocks_fetched - counters->blocks_fetched;
+ report->blks_hit =
+ rel->pgstat_info->counts.blocks_hit - counters->blocks_hit;
+}
+
+static void
+extvac_stats_start_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters)
+{
+ extvac_stats_start(rel, &counters->common);
+ counters->pages_deleted = counters->tuples_removed = 0;
+
+ if (stats != NULL)
+ {
+ /*
+ * XXX: Why do we need this code here? If it is needed, I feel lack of
+ * comments, describing the reason.
+ */
+ counters->tuples_removed = stats->tuples_removed;
+ counters->pages_deleted = stats->pages_deleted;
+ }
+}
+
+static void
+extvac_stats_end_idx(Relation rel, IndexBulkDeleteResult *stats,
+ LVExtStatCountersIdx *counters, ExtVacReport *report)
+{
+ extvac_stats_end(rel, &counters->common, report);
+ report->type = PGSTAT_EXTVAC_INDEX;
+
+ if (stats != NULL)
+ {
+ /*
+ * if something goes wrong or an user doesn't want to track a database
+ * activity - just suppress it.
+ */
+
+ /* Fill index-specific extended stats fields */
+ report->index.tuples_deleted =
+ stats->tuples_removed - counters->tuples_removed;
+ report->index.pages_deleted =
+ stats->pages_deleted - counters->pages_deleted;
+ }
+}
/*
* heap_vacuum_rel() -- perform VACUUM for one heap relation
@@ -311,6 +494,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
WalUsage startwalusage = pgWalUsage;
BufferUsage startbufferusage = pgBufferUsage;
ErrorContextCallback errcallback;
+ LVExtStatCounters extVacCounters;
+ ExtVacReport extVacReport;
char **indnames = NULL;
verbose = (params->options & VACOPT_VERBOSE) != 0;
@@ -329,7 +514,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
pgstat_progress_start_command(PROGRESS_COMMAND_VACUUM,
RelationGetRelid(rel));
-
+ extvac_stats_start(rel, &extVacCounters);
/*
* Setup error traceback support for ereport() first. The idea is to set
* up an error context callback to display additional information on any
@@ -346,6 +531,7 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->dbname = get_database_name(MyDatabaseId);
vacrel->relnamespace = get_namespace_name(RelationGetNamespace(rel));
vacrel->relname = pstrdup(RelationGetRelationName(rel));
+ vacrel->reloid = RelationGetRelid(rel);
vacrel->indname = NULL;
vacrel->phase = VACUUM_ERRCB_PHASE_UNKNOWN;
vacrel->verbose = verbose;
@@ -413,6 +599,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->lpdead_item_pages = 0;
vacrel->missed_dead_pages = 0;
vacrel->nonempty_pages = 0;
+ vacrel->set_frozen_pages = 0;
+ vacrel->set_all_visible_pages = 0;
/* dead_items_alloc allocates vacrel->dead_items later on */
/* Allocate/initialize output statistics state */
@@ -574,6 +762,20 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
vacrel->NewRelfrozenXid, vacrel->NewRelminMxid,
&frozenxid_updated, &minmulti_updated, false);
+ /* Make generic extended vacuum stats report */
+ extvac_stats_end(rel, &extVacCounters, &extVacReport);
+
+ /* Fill heap-specific extended stats fields */
+ extVacReport.type = PGSTAT_EXTVAC_HEAP;
+ extVacReport.heap.pages_scanned = vacrel->scanned_pages;
+ extVacReport.heap.pages_removed = vacrel->removed_pages;
+ extVacReport.heap.pages_frozen = vacrel->set_frozen_pages;
+ extVacReport.heap.pages_all_visible = vacrel->set_all_visible_pages;
+ extVacReport.heap.tuples_deleted = vacrel->tuples_deleted;
+ extVacReport.heap.tuples_frozen = vacrel->tuples_frozen;
+ extVacReport.heap.dead_tuples = vacrel->recently_dead_tuples + vacrel->missed_dead_tuples;
+ extVacReport.heap.index_vacuum_count = vacrel->num_index_scans;
+
/*
* Report results to the cumulative stats system, too.
*
@@ -588,7 +790,8 @@ heap_vacuum_rel(Relation rel, VacuumParams *params,
rel->rd_rel->relisshared,
Max(vacrel->new_live_tuples, 0),
vacrel->recently_dead_tuples +
- vacrel->missed_dead_tuples);
+ vacrel->missed_dead_tuples,
+ &extVacReport);
pgstat_progress_end_command();
if (instrument)
@@ -1370,6 +1573,8 @@ lazy_scan_new_or_empty(LVRelState *vacrel, Buffer buf, BlockNumber blkno,
vmbuffer, InvalidTransactionId,
VISIBILITYMAP_ALL_VISIBLE | VISIBILITYMAP_ALL_FROZEN);
END_CRIT_SECTION();
+ vacrel->set_all_visible_pages++;
+ vacrel->set_frozen_pages++;
}
freespace = PageGetHeapFreeSpace(page);
@@ -2262,11 +2467,13 @@ lazy_vacuum_heap_page(LVRelState *vacrel, BlockNumber blkno, Buffer buffer,
&all_frozen))
{
uint8 flags = VISIBILITYMAP_ALL_VISIBLE;
+ vacrel->set_all_visible_pages++;
if (all_frozen)
{
Assert(!TransactionIdIsValid(visibility_cutoff_xid));
flags |= VISIBILITYMAP_ALL_FROZEN;
+ vacrel->set_frozen_pages++;
}
PageSetAllVisible(page);
@@ -2417,6 +2624,10 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2435,6 +2646,7 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_VACUUM_INDEX,
InvalidBlockNumber, InvalidOffsetNumber);
@@ -2443,6 +2655,13 @@ lazy_vacuum_one_index(Relation indrel, IndexBulkDeleteResult *istat,
istat = vac_bulkdel_one_index(&ivinfo, istat, (void *) vacrel->dead_items,
vacrel->dead_items_info);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -2467,6 +2686,10 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
{
IndexVacuumInfo ivinfo;
LVSavedErrInfo saved_err_info;
+ LVExtStatCountersIdx extVacCounters;
+ ExtVacReport extVacReport;
+
+ extvac_stats_start_idx(indrel, istat, &extVacCounters);
ivinfo.index = indrel;
ivinfo.heaprel = vacrel->rel;
@@ -2486,12 +2709,20 @@ lazy_cleanup_one_index(Relation indrel, IndexBulkDeleteResult *istat,
*/
Assert(vacrel->indname == NULL);
vacrel->indname = pstrdup(RelationGetRelationName(indrel));
+ vacrel->indoid = RelationGetRelid(indrel);
update_vacuum_error_info(vacrel, &saved_err_info,
VACUUM_ERRCB_PHASE_INDEX_CLEANUP,
InvalidBlockNumber, InvalidOffsetNumber);
istat = vac_cleanup_one_index(&ivinfo, istat);
+ /* Make extended vacuum stats report for index */
+ extvac_stats_end_idx(indrel, istat, &extVacCounters, &extVacReport);
+
+ pgstat_report_vacuum(RelationGetRelid(indrel),
+ indrel->rd_rel->relisshared,
+ 0, 0, &extVacReport);
+
/* Revert to the previous phase information for error traceback */
restore_vacuum_error_info(vacrel, &saved_err_info);
pfree(vacrel->indname);
@@ -3101,6 +3332,8 @@ vacuum_error_callback(void *arg)
switch (errinfo->phase)
{
case VACUUM_ERRCB_PHASE_SCAN_HEAP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3116,6 +3349,8 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_HEAP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
{
if (OffsetNumberIsValid(errinfo->offnum))
@@ -3131,16 +3366,22 @@ vacuum_error_callback(void *arg)
break;
case VACUUM_ERRCB_PHASE_VACUUM_INDEX:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while vacuuming index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_INDEX_CLEANUP:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->indoid, PGSTAT_EXTVAC_INDEX);
errcontext("while cleaning up index \"%s\" of relation \"%s.%s\"",
errinfo->indname, errinfo->relnamespace, errinfo->relname);
break;
case VACUUM_ERRCB_PHASE_TRUNCATE:
+ if(geterrelevel() >= ERROR)
+ pgstat_report_vacuum_error(errinfo->reloid, PGSTAT_EXTVAC_HEAP);
if (BlockNumberIsValid(errinfo->blkno))
errcontext("while truncating relation \"%s.%s\" to %u blocks",
errinfo->relnamespace, errinfo->relname, errinfo->blkno);
diff --git a/src/backend/access/heap/visibilitymap.c b/src/backend/access/heap/visibilitymap.c
index 8b24e7bc33c..2588bb84c8b 100644
--- a/src/backend/access/heap/visibilitymap.c
+++ b/src/backend/access/heap/visibilitymap.c
@@ -96,6 +96,7 @@
#include "storage/smgr.h"
#include "utils/inval.h"
#include "utils/rel.h"
+#include "pgstat.h"
/*#define TRACE_VISIBILITYMAP */
@@ -160,6 +161,18 @@ visibilitymap_clear(Relation rel, BlockNumber heapBlk, Buffer vmbuf, uint8 flags
if (map[mapByte] & mask)
{
+ /*
+ * Initially, it didn't matter what type of flags (all-visible or frozen) we received,
+ * we just performed a reverse concatenation operation. But this information is very important
+ * for vacuum statistics. We need to find out this usingthe bit concatenation operation
+ * with the VISIBILITYMAP_ALL_VISIBLE and VISIBILITYMAP_ALL_FROZEN masks,
+ * and where the desired one matches, we increment the value there.
+ */
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_VISIBLE)
+ pgstat_count_vm_rev_all_visible(rel);
+ if (map[mapByte] >> mapOffset & flags & VISIBILITYMAP_ALL_FROZEN)
+ pgstat_count_vm_rev_all_frozen(rel);
+
map[mapByte] &= ~mask;
MarkBufferDirty(vmbuf);
diff --git a/src/backend/catalog/system_views.sql b/src/backend/catalog/system_views.sql
index 53047cab5fc..7f585f758f6 100644
--- a/src/backend/catalog/system_views.sql
+++ b/src/backend/catalog/system_views.sql
@@ -1374,3 +1374,126 @@ CREATE VIEW pg_stat_subscription_stats AS
CREATE VIEW pg_wait_events AS
SELECT * FROM pg_get_wait_events();
+--
+-- Show extended cumulative statistics on a vacuum operation over all tables and
+-- databases of the instance.
+-- Use Invalid Oid "0" as an input relation id to get stat on each table in a
+-- database.
+--
+
+CREATE VIEW pg_stats_vacuum_tables AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stats_vacuum_tables(db.oid, rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
+
+CREATE VIEW pg_stats_vacuum_indexes AS
+SELECT
+ rel.oid as relid,
+ ns.nspname AS "schema",
+ rel.relname AS relname,
+
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+
+ stats.pages_deleted,
+ stats.tuples_deleted,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ pg_stats_vacuum_indexes(db.oid, rel.oid) stats
+WHERE
+ db.datname = current_database() AND
+ rel.oid = stats.relid AND
+ ns.oid = rel.relnamespace;
+
+CREATE VIEW pg_stats_vacuum_database AS
+SELECT
+ db.oid as dboid,
+
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+
+ stats.blk_read_time,
+ stats.blk_write_time,
+
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+
+ stats.interrupts
+FROM
+ pg_database db,
+ pg_namespace ns,
+ pg_stats_vacuum_database(db.oid) stats
+WHERE
+ db.datname = current_database();
diff --git a/src/backend/commands/vacuum.c b/src/backend/commands/vacuum.c
index 48f8eab2022..ca1f4f8018c 100644
--- a/src/backend/commands/vacuum.c
+++ b/src/backend/commands/vacuum.c
@@ -103,6 +103,9 @@ pg_atomic_uint32 *VacuumSharedCostBalance = NULL;
pg_atomic_uint32 *VacuumActiveNWorkers = NULL;
int VacuumCostBalanceLocal = 0;
+/* Cumulative storage to report total vacuum delay time. */
+double VacuumDelayTime = 0; /* msec. */
+
/* non-export function prototypes */
static List *expand_vacuum_rel(VacuumRelation *vrel,
MemoryContext vac_context, int options);
@@ -2397,6 +2400,7 @@ vacuum_delay_point(void)
exit(1);
VacuumCostBalance = 0;
+ VacuumDelayTime += msec;
/*
* Balance and update limit values for autovacuum workers. We must do
diff --git a/src/backend/commands/vacuumparallel.c b/src/backend/commands/vacuumparallel.c
index f26070bff2a..dedb02963a0 100644
--- a/src/backend/commands/vacuumparallel.c
+++ b/src/backend/commands/vacuumparallel.c
@@ -1046,6 +1046,7 @@ parallel_vacuum_main(dsm_segment *seg, shm_toc *toc)
VacuumPageHit = 0;
VacuumPageMiss = 0;
VacuumPageDirty = 0;
+ VacuumDelayTime = 0.;
VacuumCostBalanceLocal = 0;
VacuumSharedCostBalance = &(shared->cost_balance);
VacuumActiveNWorkers = &(shared->active_nworkers);
diff --git a/src/backend/utils/activity/pgstat.c b/src/backend/utils/activity/pgstat.c
index dcc2ad8d954..88e4bbd1f88 100644
--- a/src/backend/utils/activity/pgstat.c
+++ b/src/backend/utils/activity/pgstat.c
@@ -127,36 +127,6 @@
#define PGSTAT_SNAPSHOT_HASH_SIZE 512
-
-/* hash table for statistics snapshots entry */
-typedef struct PgStat_SnapshotEntry
-{
- PgStat_HashKey key;
- char status; /* for simplehash use */
- void *data; /* the stats data itself */
-} PgStat_SnapshotEntry;
-
-
-/* ----------
- * Backend-local Hash Table Definitions
- * ----------
- */
-
-/* for stats snapshot entries */
-#define SH_PREFIX pgstat_snapshot
-#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
-#define SH_KEY_TYPE PgStat_HashKey
-#define SH_KEY key
-#define SH_HASH_KEY(tb, key) \
- pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
-#define SH_EQUAL(tb, a, b) \
- pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
-#define SH_SCOPE static inline
-#define SH_DEFINE
-#define SH_DECLARE
-#include "lib/simplehash.h"
-
-
/* ----------
* Local function forward declarations
* ----------
@@ -170,7 +140,7 @@ static void pgstat_reset_after_failure(void);
static bool pgstat_flush_pending_entries(bool nowait);
static void pgstat_prep_snapshot(void);
-static void pgstat_build_snapshot(void);
+static void pgstat_build_snapshot(PgStat_Kind statKind);
static void pgstat_build_snapshot_fixed(PgStat_Kind kind);
static inline bool pgstat_is_kind_valid(int ikind);
@@ -764,6 +734,56 @@ pgstat_reset_of_kind(PgStat_Kind kind)
pgstat_reset_entries_of_kind(kind, ts);
}
+void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info)
+{
+ dst->total_blks_read += src->total_blks_read;
+ dst->total_blks_hit += src->total_blks_hit;
+ dst->total_blks_dirtied += src->total_blks_dirtied;
+ dst->total_blks_written += src->total_blks_written;
+ dst->wal_bytes += src->wal_bytes;
+ dst->wal_fpi += src->wal_fpi;
+ dst->wal_records += src->wal_records;
+ dst->blk_read_time += src->blk_read_time;
+ dst->blk_write_time += src->blk_write_time;
+ dst->delay_time += src->delay_time;
+ dst->system_time += src->system_time;
+ dst->user_time += src->user_time;
+ dst->total_time += src->total_time;
+ dst->interrupts += src->interrupts;
+
+ if (!accumulate_reltype_specific_info)
+ return;
+
+ if (dst->type == PGSTAT_EXTVAC_INVALID)
+ dst->type = src->type;
+
+ Assert(src->type == PGSTAT_EXTVAC_INVALID || src->type == dst->type);
+
+ if (dst->type == src->type)
+ {
+ dst->blks_fetched += src->blks_fetched;
+ dst->blks_hit += src->blks_hit;
+
+ if (dst->type == PGSTAT_EXTVAC_HEAP)
+ {
+ dst->heap.pages_scanned += src->heap.pages_scanned;
+ dst->heap.pages_removed += src->heap.pages_removed;
+ dst->heap.pages_frozen += src->heap.pages_frozen;
+ dst->heap.pages_all_visible += src->heap.pages_all_visible;
+ dst->heap.tuples_deleted += src->heap.tuples_deleted;
+ dst->heap.tuples_frozen += src->heap.tuples_frozen;
+ dst->heap.dead_tuples += src->heap.dead_tuples;
+ dst->heap.index_vacuum_count += src->heap.index_vacuum_count;
+ }
+ else if (dst->type == PGSTAT_EXTVAC_INDEX)
+ {
+ dst->index.pages_deleted += src->index.pages_deleted;
+ dst->index.tuples_deleted += src->index.tuples_deleted;
+ }
+ }
+}
/* ------------------------------------------------------------
* Fetching of stats
@@ -828,7 +848,7 @@ pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid)
/* if we need to build a full snapshot, do so */
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
/* if caching is desired, look up in cache */
if (pgstat_fetch_consistency > PGSTAT_FETCH_CONSISTENCY_NONE)
@@ -944,7 +964,7 @@ pgstat_snapshot_fixed(PgStat_Kind kind)
pgstat_clear_snapshot();
if (pgstat_fetch_consistency == PGSTAT_FETCH_CONSISTENCY_SNAPSHOT)
- pgstat_build_snapshot();
+ pgstat_build_snapshot(PGSTAT_KIND_INVALID);
else
pgstat_build_snapshot_fixed(kind);
@@ -972,8 +992,33 @@ pgstat_prep_snapshot(void)
NULL);
}
+
+/*
+ * Trivial external interface to build a snapshot for table statistics only.
+ */
+void
+pgstat_update_snapshot(PgStat_Kind kind)
+{
+ int save_consistency_guc = pgstat_fetch_consistency;
+ pgstat_clear_snapshot();
+
+ PG_TRY();
+ {
+ pgstat_fetch_consistency = PGSTAT_FETCH_CONSISTENCY_SNAPSHOT;
+ if (kind == PGSTAT_KIND_RELATION)
+ pgstat_build_snapshot(PGSTAT_KIND_RELATION);
+ else if (kind == PGSTAT_KIND_DATABASE)
+ pgstat_build_snapshot(PGSTAT_KIND_DATABASE);
+ }
+ PG_FINALLY();
+ {
+ pgstat_fetch_consistency = save_consistency_guc;
+ }
+ PG_END_TRY();
+}
+
static void
-pgstat_build_snapshot(void)
+pgstat_build_snapshot(PgStat_Kind statKind)
{
dshash_seq_status hstat;
PgStatShared_HashEntry *p;
@@ -1018,6 +1063,10 @@ pgstat_build_snapshot(void)
if (p->dropped)
continue;
+ if (statKind != PGSTAT_KIND_INVALID && statKind != p->key.kind)
+ /* Load stat of specific type, if defined */
+ continue;
+
Assert(pg_atomic_read_u32(&p->refcount) > 0);
stats_data = dsa_get_address(pgStatLocal.dsa, p->body);
diff --git a/src/backend/utils/activity/pgstat_database.c b/src/backend/utils/activity/pgstat_database.c
index 29bc0909748..a060d1a4042 100644
--- a/src/backend/utils/activity/pgstat_database.c
+++ b/src/backend/utils/activity/pgstat_database.c
@@ -430,6 +430,7 @@ pgstat_database_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
pgstat_unlock_entry(entry_ref);
memset(pendingent, 0, sizeof(*pendingent));
+ memset(&(pendingent)->vacuum_ext, 0, sizeof(ExtVacReport));
return true;
}
diff --git a/src/backend/utils/activity/pgstat_relation.c b/src/backend/utils/activity/pgstat_relation.c
index 8a3f7d434cf..cc09aba571f 100644
--- a/src/backend/utils/activity/pgstat_relation.c
+++ b/src/backend/utils/activity/pgstat_relation.c
@@ -204,16 +204,51 @@ pgstat_drop_relation(Relation rel)
}
}
+/* ---------
+ * pgstat_report_vacuum_error() -
+ *
+ * Tell the collector about an (auto)vacuum interruption.
+ * ---------
+ */
+void
+pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type)
+{
+ PgStat_EntryRef *entry_ref;
+ PgStatShared_Relation *shtabentry;
+ PgStat_StatTabEntry *tabentry;
+ Oid dboid = MyDatabaseId;
+ PgStat_StatDBEntry *dbentry; /* pending database entry */
+
+ if (!pgstat_track_counts)
+ return;
+
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_RELATION,
+ dboid, tableoid, false);
+
+ shtabentry = (PgStatShared_Relation *) entry_ref->shared_stats;
+ tabentry = &shtabentry->stats;
+
+ tabentry->vacuum_ext.interrupts++;
+ tabentry->vacuum_ext.type = m_type;
+ pgstat_unlock_entry(entry_ref);
+
+ dbentry = pgstat_prep_database_pending(dboid);
+ dbentry->vacuum_ext.interrupts++;
+ dbentry->vacuum_ext.type = m_type;
+}
+
/*
* Report that the table was just vacuumed and flush IO statistics.
*/
void
pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples)
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params)
{
PgStat_EntryRef *entry_ref;
PgStatShared_Relation *shtabentry;
PgStat_StatTabEntry *tabentry;
+ PgStatShared_Database *dbentry;
Oid dboid = (shared ? InvalidOid : MyDatabaseId);
TimestampTz ts;
@@ -233,6 +268,8 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
tabentry->live_tuples = livetuples;
tabentry->dead_tuples = deadtuples;
+ pgstat_accumulate_extvac_stats(&tabentry->vacuum_ext, params, true);
+
/*
* It is quite possible that a non-aggressive VACUUM ended up skipping
* various pages, however, we'll zero the insert counter here regardless.
@@ -265,6 +302,16 @@ pgstat_report_vacuum(Oid tableoid, bool shared,
* VACUUM command has processed all tables and committed.
*/
pgstat_flush_io(false);
+ if (dboid != InvalidOid)
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dboid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ pgstat_accumulate_extvac_stats(&dbentry->stats.vacuum_ext, params, false);
+ pgstat_unlock_entry(entry_ref);
+ }
+
}
/*
@@ -861,6 +908,9 @@ pgstat_relation_flush_cb(PgStat_EntryRef *entry_ref, bool nowait)
tabentry->blocks_fetched += lstats->counts.blocks_fetched;
tabentry->blocks_hit += lstats->counts.blocks_hit;
+ tabentry->rev_all_frozen_pages += lstats->counts.rev_all_frozen_pages;
+ tabentry->rev_all_visible_pages += lstats->counts.rev_all_visible_pages;
+
/* Clamp live_tuples in case of negative delta_live_tuples */
tabentry->live_tuples = Max(tabentry->live_tuples, 0);
/* Likewise for dead_tuples */
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 3876339ee1b..a778e5b2fec 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -31,6 +31,7 @@
#include "utils/acl.h"
#include "utils/builtins.h"
#include "utils/timestamp.h"
+#include "utils/pgstat_internal.h"
#define UINT32_ACCESS_ONCE(var) ((uint32)(*((volatile uint32 *)&(var))))
@@ -2032,3 +2033,292 @@ pg_stat_have_stats(PG_FUNCTION_ARGS)
PG_RETURN_BOOL(pgstat_have_entry(kind, dboid, objoid));
}
+
+#define EXTVACHEAPSTAT_COLUMNS 27
+#define EXTVACIDXSTAT_COLUMNS 19
+#define EXTVACDBSTAT_COLUMNS 15
+#define EXTVACSTAT_COLUMNS Max(EXTVACHEAPSTAT_COLUMNS, EXTVACIDXSTAT_COLUMNS)
+
+static Oid CurrentDatabaseId = InvalidOid;
+
+
+/*
+ * Fetch stat collector data for specific database and table, which loading from disc.
+ * It is maybe expensive, but i guess we won't use that machinery often.
+ * The kind of bufferization is based on CurrentDatabaseId value.
+ */
+static PgStat_StatTabEntry *
+fetch_dbstat_tabentry(Oid dbid, Oid relid)
+{
+ Oid storedMyDatabaseId = MyDatabaseId;
+ PgStat_StatTabEntry *tabentry = NULL;
+
+ if (OidIsValid(CurrentDatabaseId) && CurrentDatabaseId == dbid)
+ /* Quick path when we read data from the same database */
+ return pgstat_fetch_stat_tabentry(relid);
+
+ pgstat_clear_snapshot();
+
+ /* Tricky turn here: enforce pgstat to think that our database us dbid */
+
+ MyDatabaseId = dbid;
+
+ PG_TRY();
+ {
+ tabentry = pgstat_fetch_stat_tabentry(relid);
+ MyDatabaseId = storedMyDatabaseId;
+ }
+ PG_CATCH();
+ {
+ MyDatabaseId = storedMyDatabaseId;
+ }
+ PG_END_TRY();
+
+ return tabentry;
+}
+
+static void
+tuplestore_put_for_database(Oid dbid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStatShared_Database *dbentry, int ncolumns)
+{
+ Datum values[EXTVACDBSTAT_COLUMNS];
+ bool nulls[EXTVACDBSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACDBSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(dbid);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(dbentry->stats.vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, dbentry->stats.vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(dbentry->stats.vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(dbentry->stats.vacuum_ext.interrupts);
+
+
+ Assert(i == ncolumns);
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+static void
+tuplestore_put_for_relation(Oid relid, Tuplestorestate *tupstore,
+ TupleDesc tupdesc, PgStat_StatTabEntry *tabentry, int ncolumns)
+{
+ Datum values[EXTVACSTAT_COLUMNS];
+ bool nulls[EXTVACSTAT_COLUMNS];
+ char buf[256];
+ int i = 0;
+
+ memset(nulls, 0, EXTVACSTAT_COLUMNS * sizeof(bool));
+
+ values[i++] = ObjectIdGetDatum(relid);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_read);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_dirtied);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.total_blks_written);
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_fetched -
+ tabentry->vacuum_ext.blks_hit);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.blks_hit);
+
+ if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_HEAP)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_scanned);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_removed);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.pages_all_visible);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.tuples_frozen);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.dead_tuples);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.heap.index_vacuum_count);
+ values[i++] = Int64GetDatum(tabentry->rev_all_frozen_pages);
+ values[i++] = Int64GetDatum(tabentry->rev_all_visible_pages);
+
+ }
+ else if (tabentry->vacuum_ext.type == PGSTAT_EXTVAC_INDEX)
+ {
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.pages_deleted);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.index.tuples_deleted);
+ }
+
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_records);
+ values[i++] = Int64GetDatum(tabentry->vacuum_ext.wal_fpi);
+
+ /* Convert to numeric, like pg_stat_statements */
+ snprintf(buf, sizeof buf, UINT64_FORMAT, tabentry->vacuum_ext.wal_bytes);
+ values[i++] = DirectFunctionCall3(numeric_in,
+ CStringGetDatum(buf),
+ ObjectIdGetDatum(0),
+ Int32GetDatum(-1));
+
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_read_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.blk_write_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.delay_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.system_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.user_time);
+ values[i++] = Float8GetDatum(tabentry->vacuum_ext.total_time);
+ values[i++] = Int32GetDatum(tabentry->vacuum_ext.interrupts);
+
+ Assert(i == ncolumns);
+
+ tuplestore_putvalues(tupstore, tupdesc, values, nulls);
+}
+
+/*
+ * Get the vacuum statistics for the heap tables or indexes.
+ */
+static Datum
+pg_stats_vacuum(FunctionCallInfo fcinfo, ExtVacReportType type, int ncolumns)
+{
+ ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
+ MemoryContext per_query_ctx;
+ MemoryContext oldcontext;
+ Tuplestorestate *tupstore;
+ TupleDesc tupdesc;
+ Oid dbid = PG_GETARG_OID(0);
+ PgStat_StatTabEntry *tabentry;
+
+ InitMaterializedSRF(fcinfo, 0);
+
+ /* Check if caller supports us returning a tuplestore */
+ if (rsinfo == NULL || !IsA(rsinfo, ReturnSetInfo))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("set-valued function called in context that cannot accept a set")));
+ /* Switch to long-lived context to create the returned data structures */
+ per_query_ctx = rsinfo->econtext->ecxt_per_query_memory;
+ oldcontext = MemoryContextSwitchTo(per_query_ctx);
+
+ /* Build a tuple descriptor for our result type */
+ if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
+ elog(ERROR, "return type must be a row type");
+
+ Assert(tupdesc->natts == ncolumns);
+
+ tupstore = tuplestore_begin_heap(true, false, work_mem);
+ Assert (tupstore != NULL);
+ rsinfo->setResult = tupstore;
+ rsinfo->setDesc = tupdesc;
+
+ MemoryContextSwitchTo(oldcontext);
+
+ if (type == PGSTAT_EXTVAC_INDEX || type == PGSTAT_EXTVAC_HEAP)
+ {
+ Oid relid = PG_GETARG_OID(1);
+
+ /* Load table statistics for specified database. */
+ if (OidIsValid(relid))
+ {
+ tabentry = fetch_dbstat_tabentry(dbid, relid);
+ if (tabentry == NULL || tabentry->vacuum_ext.type != type)
+ /* Table don't exists or isn't an heap relation. */
+ PG_RETURN_NULL();
+
+ tuplestore_put_for_relation(relid, tupstore, tupdesc, tabentry, ncolumns);
+ }
+ else
+ {
+ SnapshotIterator hashiter;
+ PgStat_SnapshotEntry *entry;
+ Oid storedMyDatabaseId = MyDatabaseId;
+
+ pgstat_update_snapshot(PGSTAT_KIND_RELATION);
+ MyDatabaseId = storedMyDatabaseId;
+
+
+ /* Iterate the snapshot */
+ InitSnapshotIterator(pgStatLocal.snapshot.stats, &hashiter);
+
+ while ((entry = ScanStatSnapshot(pgStatLocal.snapshot.stats, &hashiter)) != NULL)
+ {
+ Oid reloid;
+
+ CHECK_FOR_INTERRUPTS();
+
+ tabentry = (PgStat_StatTabEntry *) entry->data;
+ reloid = entry->key.objoid;
+
+ if (tabentry != NULL && tabentry->vacuum_ext.type == type)
+ tuplestore_put_for_relation(reloid, tupstore, tupdesc, tabentry, ncolumns);
+ }
+ }
+ }
+ else if (type == PGSTAT_EXTVAC_DB)
+ {
+ PgStatShared_Database *dbentry;
+ PgStat_EntryRef *entry_ref;
+ Oid storedMyDatabaseId = MyDatabaseId;
+
+ pgstat_update_snapshot(PGSTAT_KIND_DATABASE);
+ MyDatabaseId = storedMyDatabaseId;
+
+ if (OidIsValid(dbid))
+ {
+ entry_ref = pgstat_get_entry_ref_locked(PGSTAT_KIND_DATABASE,
+ dbid, InvalidOid, false);
+ dbentry = (PgStatShared_Database *) entry_ref->shared_stats;
+
+ if (dbentry == NULL)
+ /* Table doesn't exist or isn't a heap relation */
+ PG_RETURN_NULL();
+ tuplestore_put_for_database(dbid, tupstore, tupdesc, dbentry, ncolumns);
+ pgstat_unlock_entry(entry_ref);
+ }
+ else
+ PG_RETURN_NULL();
+ }
+ PG_RETURN_NULL();
+}
+
+/*
+ * Get the vacuum statistics for the heap tables.
+ */
+Datum
+pg_stats_vacuum_tables(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_HEAP, EXTVACHEAPSTAT_COLUMNS);
+
+ PG_RETURN_NULL();
+}
+
+/*
+ * Get the vacuum statistics for the indexes.
+ */
+Datum
+pg_stats_vacuum_indexes(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_INDEX, EXTVACIDXSTAT_COLUMNS);
+
+ PG_RETURN_NULL();
+}
+
+/*
+ * Get the vacuum statistics for the database.
+ */
+Datum
+pg_stats_vacuum_database(PG_FUNCTION_ARGS)
+{
+ return pg_stats_vacuum(fcinfo, PGSTAT_EXTVAC_DB, EXTVACDBSTAT_COLUMNS);
+
+ PG_RETURN_NULL();
+}
diff --git a/src/backend/utils/error/elog.c b/src/backend/utils/error/elog.c
index d91a85cb2d7..3cee3436d9f 100644
--- a/src/backend/utils/error/elog.c
+++ b/src/backend/utils/error/elog.c
@@ -1602,6 +1602,19 @@ getinternalerrposition(void)
return edata->internalpos;
}
+/*
+ * Return elevel of errors
+ */
+int
+geterrelevel(void)
+{
+ ErrorData *edata = &errordata[errordata_stack_depth];
+
+ /* we don't bother incrementing recursion_depth */
+ CHECK_STACK_DEPTH();
+
+ return edata->elevel;
+}
/*
* Functions to allow construction of error message strings separately from
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 6a5476d3c4c..36c691986c0 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -12184,5 +12184,31 @@
proallargtypes => '{int8,pg_lsn,pg_lsn,int4}', proargmodes => '{o,o,o,o}',
proargnames => '{summarized_tli,summarized_lsn,pending_lsn,summarizer_pid}',
prosrc => 'pg_get_wal_summarizer_state' },
-
+{ oid => '4701',
+ descr => 'pg_stats_vacuum_tables return stats values',
+ proname => 'pg_stats_vacuum_tables', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid oid',
+ proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_scanned,pages_removed,pages_frozen,pages_all_visible,tuples_deleted,tuples_frozen,dead_tuples,index_vacuum_count,rev_all_frozen_pages,rev_all_visible_pages,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stats_vacuum_tables' },
+{ oid => '4702',
+ descr => 'pg_stats_vacuum_indexes return stats values',
+ proname => 'pg_stats_vacuum_indexes', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid oid',
+ proallargtypes => '{oid,oid,oid,int8,int8,int8,int8,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dboid,reloid,relid,total_blks_read,total_blks_hit,total_blks_dirtied,total_blks_written,rel_blks_read,rel_blks_hit,pages_deleted,tuples_deleted,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stats_vacuum_indexes' },
+ { oid => '4703',
+ descr => 'pg_stats_vacuum_database return stats values',
+ proname => 'pg_stats_vacuum_database', provolatile => 's', prorettype => 'record',proisstrict => 'f',
+ proretset => 't',
+ proargtypes => 'oid',
+ proallargtypes => '{oid,oid,int8,int8,int8,int8,int8,int8,numeric,float8,float8,float8,float8,float8,float8,int4}',
+ proargmodes => '{i,o,o,o,o,o,o,o,o,o,o,o,o,o,o,o}',
+ proargnames => '{dboid,dboid,db_blks_read,db_blks_hit,total_blks_dirtied,total_blks_written,wal_records,wal_fpi,wal_bytes,blk_read_time,blk_write_time,delay_time,system_time,user_time,total_time,interrupts}',
+ prosrc => 'pg_stats_vacuum_database' },
]
diff --git a/src/include/commands/vacuum.h b/src/include/commands/vacuum.h
index 759f9a87d38..07b28b15d9f 100644
--- a/src/include/commands/vacuum.h
+++ b/src/include/commands/vacuum.h
@@ -308,6 +308,7 @@ extern PGDLLIMPORT int vacuum_multixact_failsafe_age;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumSharedCostBalance;
extern PGDLLIMPORT pg_atomic_uint32 *VacuumActiveNWorkers;
extern PGDLLIMPORT int VacuumCostBalanceLocal;
+extern PGDLLIMPORT double VacuumDelayTime;
extern PGDLLIMPORT bool VacuumFailsafeActive;
extern PGDLLIMPORT double vacuum_cost_delay;
diff --git a/src/include/pgstat.h b/src/include/pgstat.h
index 2136239710e..b208e530826 100644
--- a/src/include/pgstat.h
+++ b/src/include/pgstat.h
@@ -137,6 +137,86 @@ typedef struct PgStat_BackendSubEntry
PgStat_Counter sync_error_count;
} PgStat_BackendSubEntry;
+
+/* Type of ExtVacReport */
+typedef enum ExtVacReportType
+{
+ PGSTAT_EXTVAC_INVALID = 0,
+ PGSTAT_EXTVAC_HEAP = 1,
+ PGSTAT_EXTVAC_INDEX = 2,
+ PGSTAT_EXTVAC_DB = 3,
+} ExtVacReportType;
+
+/* ----------
+ *
+ * ExtVacReport
+ *
+ * Additional statistics of vacuum processing over a relation.
+ * pages_removed is the amount by which the physically shrank,
+ * if any (ie the change in its total size on disk)
+ * pages_deleted refer to free space within the index file
+ * ----------
+ */
+typedef struct ExtVacReport
+{
+ int64 total_blks_read; /* number of pages that were missed in shared buffers during a vacuum of specific relation */
+ int64 total_blks_hit; /* number of pages that were found in shared buffers during a vacuum of specific relation */
+ int64 total_blks_dirtied; /* number of pages marked as 'Dirty' during a vacuum of specific relation. */
+ int64 total_blks_written; /* number of pages written during a vacuum of specific relation. */
+
+ int64 blks_fetched; /* number of a relation blocks, fetched during the vacuum. */
+ int64 blks_hit; /* number of a relation blocks, found in shared buffers during the vacuum. */
+
+ /* Vacuum WAL usage stats */
+ int64 wal_records; /* wal usage: number of WAL records */
+ int64 wal_fpi; /* wal usage: number of WAL full page images produced */
+ uint64 wal_bytes; /* wal usage: size of WAL records produced */
+
+ /* Time stats. */
+ double blk_read_time; /* time spent reading pages, in msec */
+ double blk_write_time; /* time spent writing pages, in msec */
+ double delay_time; /* how long vacuum slept in vacuum delay point, in msec */
+ double system_time; /* amount of time the CPU was busy executing vacuum code in kernel space, in msec */
+ double user_time; /* amount of time the CPU was busy executing vacuum code in user space, in msec */
+ double total_time; /* total time of a vacuum operation, in msec */
+
+ /* Interruptions on any errors. */
+ int32 interrupts;
+
+ ExtVacReportType type; /* heap, index, etc. */
+
+ /* ----------
+ *
+ * There are separate metrics of statistic for tables and indexes,
+ * which collect during vacuum.
+ * The union operator allows to combine these statistics
+ * so that each metric is assigned to a specific class of collected statistics.
+ * Such a combined structure was called per_type_stats.
+ * The name of the structure itself is not used anywhere,
+ * it exists only for understanding the code.
+ * ----------
+ */
+ union
+ {
+ struct
+ {
+ int64 pages_scanned; /* number of pages we examined */
+ int64 pages_removed; /* number of pages removed by vacuum */
+ int64 pages_frozen; /* number of pages marked in VM as frozen */
+ int64 pages_all_visible; /* number of pages marked in VM as all-visible */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ int64 tuples_frozen; /* tuples frozen up by vacuum */
+ int64 dead_tuples; /* number of deleted tuples which vacuum cannot clean up by vacuum operation */
+ int64 index_vacuum_count; /* number of index vacuumings */
+ } heap;
+ struct
+ {
+ int64 pages_deleted; /* number of pages deleted by vacuum */
+ int64 tuples_deleted; /* tuples deleted by vacuum */
+ } index;
+ } /* per_type_stats */;
+} ExtVacReport;
+
/* ----------
* PgStat_TableCounts The actual per-table counts kept by a backend
*
@@ -177,6 +257,16 @@ typedef struct PgStat_TableCounts
PgStat_Counter blocks_fetched;
PgStat_Counter blocks_hit;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ /*
+ * Additional cumulative stat on vacuum operations.
+ * Use an expensive structure as an abstraction for different types of
+ * relations.
+ */
+ ExtVacReport vacuum_ext;
} PgStat_TableCounts;
/* ----------
@@ -235,7 +325,7 @@ typedef struct PgStat_TableXactStatus
* ------------------------------------------------------------
*/
-#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAC
+#define PGSTAT_FILE_FORMAT_ID 0x01A5BCAD
typedef struct PgStat_ArchiverStats
{
@@ -354,6 +444,8 @@ typedef struct PgStat_StatDBEntry
PgStat_Counter sessions_killed;
TimestampTz stat_reset_timestamp;
+
+ ExtVacReport vacuum_ext; /* extended vacuum statistics */
} PgStat_StatDBEntry;
typedef struct PgStat_StatFuncEntry
@@ -426,6 +518,11 @@ typedef struct PgStat_StatTabEntry
PgStat_Counter analyze_count;
TimestampTz last_autoanalyze_time; /* autovacuum initiated */
PgStat_Counter autoanalyze_count;
+
+ PgStat_Counter rev_all_visible_pages;
+ PgStat_Counter rev_all_frozen_pages;
+
+ ExtVacReport vacuum_ext;
} PgStat_StatTabEntry;
typedef struct PgStat_WalStats
@@ -591,10 +688,12 @@ extern void pgstat_assoc_relation(Relation rel);
extern void pgstat_unlink_relation(Relation rel);
extern void pgstat_report_vacuum(Oid tableoid, bool shared,
- PgStat_Counter livetuples, PgStat_Counter deadtuples);
+ PgStat_Counter livetuples, PgStat_Counter deadtuples,
+ ExtVacReport *params);
extern void pgstat_report_analyze(Relation rel,
PgStat_Counter livetuples, PgStat_Counter deadtuples,
bool resetcounter);
+extern void pgstat_report_vacuum_error(Oid tableoid, ExtVacReportType m_type);
/*
* If stats are enabled, but pending data hasn't been prepared yet, call
@@ -642,6 +741,17 @@ extern void pgstat_report_analyze(Relation rel,
if (pgstat_should_count_relation(rel)) \
(rel)->pgstat_info->counts.blocks_hit++; \
} while (0)
+/* accumulate unfrozen all-visible and all-frozen pages */
+#define pgstat_count_vm_rev_all_visible(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_visible_pages++; \
+ } while (0)
+#define pgstat_count_vm_rev_all_frozen(rel) \
+ do { \
+ if (pgstat_should_count_relation(rel)) \
+ (rel)->pgstat_info->counts.rev_all_frozen_pages++; \
+ } while (0)
extern void pgstat_count_heap_insert(Relation rel, PgStat_Counter n);
extern void pgstat_count_heap_update(Relation rel, bool hot, bool newpage);
@@ -658,7 +768,9 @@ extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry(Oid relid);
extern PgStat_StatTabEntry *pgstat_fetch_stat_tabentry_ext(bool shared,
Oid reloid);
extern PgStat_TableStatus *find_tabstat_entry(Oid rel_id);
-
+extern void
+pgstat_accumulate_extvac_stats(ExtVacReport *dst, ExtVacReport *src,
+ bool accumulate_reltype_specific_info);
/*
* Functions in pgstat_replslot.c
diff --git a/src/include/utils/elog.h b/src/include/utils/elog.h
index 054dd2bf62f..c6225b9cddd 100644
--- a/src/include/utils/elog.h
+++ b/src/include/utils/elog.h
@@ -228,7 +228,7 @@ extern int err_generic_string(int field, const char *str);
extern int geterrcode(void);
extern int geterrposition(void);
extern int getinternalerrposition(void);
-
+extern int geterrelevel(void);
/*----------
* Old-style error reporting API: to be used in this way:
diff --git a/src/include/utils/pgstat_internal.h b/src/include/utils/pgstat_internal.h
index dbbca316025..e6079c67421 100644
--- a/src/include/utils/pgstat_internal.h
+++ b/src/include/utils/pgstat_internal.h
@@ -516,7 +516,7 @@ extern PgStat_EntryRef *pgstat_fetch_pending_entry(PgStat_Kind kind, Oid dboid,
extern void *pgstat_fetch_entry(PgStat_Kind kind, Oid dboid, Oid objoid);
extern void pgstat_snapshot_fixed(PgStat_Kind kind);
-
+extern void pgstat_update_snapshot(PgStat_Kind kind);
/*
* Functions in pgstat_archiver.c
@@ -805,4 +805,38 @@ pgstat_get_entry_data(PgStat_Kind kind, PgStatShared_Common *entry)
return ((char *) (entry)) + off;
}
+/* hash table for statistics snapshots entry */
+typedef struct PgStat_SnapshotEntry
+{
+ PgStat_HashKey key;
+ char status; /* for simplehash use */
+ void *data; /* the stats data itself */
+} PgStat_SnapshotEntry;
+
+/* ----------
+ * Backend-local Hash Table Definitions
+ * ----------
+ */
+
+/* for stats snapshot entries */
+#define SH_PREFIX pgstat_snapshot
+#define SH_ELEMENT_TYPE PgStat_SnapshotEntry
+#define SH_KEY_TYPE PgStat_HashKey
+#define SH_KEY key
+#define SH_HASH_KEY(tb, key) \
+ pgstat_hash_hash_key(&key, sizeof(PgStat_HashKey), NULL)
+#define SH_EQUAL(tb, a, b) \
+ pgstat_cmp_hash_key(&a, &b, sizeof(PgStat_HashKey), NULL) == 0
+#define SH_SCOPE static inline
+#define SH_DEFINE
+#define SH_DECLARE
+#include "lib/simplehash.h"
+
+typedef pgstat_snapshot_iterator SnapshotIterator;
+
+#define InitSnapshotIterator(htable, iter) \
+ pgstat_snapshot_start_iterate(htable, iter);
+#define ScanStatSnapshot(htable, iter) \
+ pgstat_snapshot_iterate(htable, iter)
+
#endif /* PGSTAT_INTERNAL_H */
diff --git a/src/test/isolation/expected/vacuum-extended-statistic.out b/src/test/isolation/expected/vacuum-extended-statistic.out
new file mode 100644
index 00000000000..83333b7dd27
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extended-statistic.out
@@ -0,0 +1,419 @@
+Parsed test spec with 1 sessions
+
+starting permutation: s1_insert s1_print_vacuum_stats_tables s1_print_vacuum_stats_indexes s1_set_agressive_vacuum s1_analyze s1_delete_half_table s1_print_vacuum_stats_tables s1_print_vacuum_stats_indexes s1_checkpoint s1_vacuum s1_print_vacuum_stats_tables s1_print_vacuum_stats_indexes s1_delete_full_table s1_checkpoint s1_vacuum_parallel s1_print_vacuum_stats_tables s1_print_vacuum_stats_indexes s1_insert s1_update s1_checkpoint s1_vacuum_full s1_print_vacuum_stats_tables s1_print_vacuum_stats_indexes s1_checkpoint s1_delete_full_table s1_trancate s1_checkpoint s1_vacuum s1_print_vacuum_stats_tables s1_print_vacuum_stats_indexes
+step s1_insert: INSERT INTO vestat(x) SELECT id FROM generate_series(1,770) As id;
+step s1_print_vacuum_stats_tables:
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+
+relname|pages_frozen|tuples_deleted|pages_scanned|pages_removed
+-------+------------+--------------+-------------+-------------
+(0 rows)
+
+step s1_print_vacuum_stats_indexes:
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+
+relname|pages_deleted|tuples_deleted
+-------+-------------+--------------
+(0 rows)
+
+step s1_set_agressive_vacuum: SET vacuum_freeze_min_age = 0;
+step s1_analyze: ANALYZE vestat;
+step s1_delete_half_table: DELETE FROM vestat WHERE x % 2 = 0;
+step s1_print_vacuum_stats_tables:
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+
+relname|pages_frozen|tuples_deleted|pages_scanned|pages_removed
+-------+------------+--------------+-------------+-------------
+(0 rows)
+
+step s1_print_vacuum_stats_indexes:
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+
+relname|pages_deleted|tuples_deleted
+-------+-------------+--------------
+(0 rows)
+
+step s1_checkpoint: CHECKPOINT;
+step s1_vacuum: VACUUM vestat;
+step s1_print_vacuum_stats_tables:
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+
+relname|pages_frozen|tuples_deleted|pages_scanned|pages_removed
+-------+------------+--------------+-------------+-------------
+vestat | 4| 385| 4| 0
+(1 row)
+
+step s1_print_vacuum_stats_indexes:
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+
+relname |pages_deleted|tuples_deleted
+-----------+-------------+--------------
+vestat_pkey| 0| 385
+(1 row)
+
+step s1_delete_full_table: DELETE FROM vestat;
+step s1_checkpoint: CHECKPOINT;
+step s1_vacuum_parallel: VACUUM (PARALLEL 2, INDEX_CLEANUP ON) vestat;
+step s1_print_vacuum_stats_tables:
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+
+relname|pages_frozen|tuples_deleted|pages_scanned|pages_removed
+-------+------------+--------------+-------------+-------------
+vestat | 8| 770| 8| 4
+(1 row)
+
+step s1_print_vacuum_stats_indexes:
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+
+relname |pages_deleted|tuples_deleted
+-----------+-------------+--------------
+vestat_pkey| 1| 770
+(1 row)
+
+step s1_insert: INSERT INTO vestat(x) SELECT id FROM generate_series(1,770) As id;
+step s1_update: UPDATE vestat SET x = x+1;
+ERROR: duplicate key value violates unique constraint "vestat_pkey"
+step s1_checkpoint: CHECKPOINT;
+step s1_vacuum_full: VACUUM FULL vestat;
+step s1_print_vacuum_stats_tables:
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+
+relname|pages_frozen|tuples_deleted|pages_scanned|pages_removed
+-------+------------+--------------+-------------+-------------
+vestat | 8| 770| 8| 4
+(1 row)
+
+step s1_print_vacuum_stats_indexes:
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+
+relname |pages_deleted|tuples_deleted
+-----------+-------------+--------------
+vestat_pkey| 1| 770
+(1 row)
+
+step s1_checkpoint: CHECKPOINT;
+step s1_delete_full_table: DELETE FROM vestat;
+step s1_trancate: TRUNCATE vestat;
+step s1_checkpoint: CHECKPOINT;
+step s1_vacuum: VACUUM vestat;
+step s1_print_vacuum_stats_tables:
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+
+relname|pages_frozen|tuples_deleted|pages_scanned|pages_removed
+-------+------------+--------------+-------------+-------------
+vestat | 8| 770| 8| 4
+(1 row)
+
+step s1_print_vacuum_stats_indexes:
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+
+relname |pages_deleted|tuples_deleted
+-----------+-------------+--------------
+vestat_pkey| 1| 770
+(1 row)
+
+
+starting permutation: s1_insert s1_set_agressive_vacuum s1_analyze s1_delete_half_table s1_checkpoint s1_difference s1_save_walls s1_checkpoint s1_vacuum s1_checkpoint s1_difference s1_save_walls s1_delete_full_table s1_checkpoint s1_vacuum_parallel s1_checkpoint s1_difference s1_save_walls s1_insert s1_update s1_checkpoint s1_vacuum_full s1_checkpoint s1_difference s1_save_walls s1_checkpoint s1_delete_full_table s1_trancate s1_vacuum s1_checkpoint s1_difference s1_save_walls
+step s1_insert: INSERT INTO vestat(x) SELECT id FROM generate_series(1,770) As id;
+step s1_set_agressive_vacuum: SET vacuum_freeze_min_age = 0;
+step s1_analyze: ANALYZE vestat;
+step s1_delete_half_table: DELETE FROM vestat WHERE x % 2 = 0;
+step s1_checkpoint: CHECKPOINT;
+step s1_difference:
+ SELECT t1.wal_records - t0.wal_records > 0 AS dWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS dFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS dWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+ SELECT t1.wal_records - t0.wal_records > 0 AS iWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS iFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS iWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+dwr|dfpi|dwb
+---+----+---
+(0 rows)
+
+iwr|ifpi|iwb
+---+----+---
+(0 rows)
+
+step s1_save_walls:
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = dWR, wal_fpi = dFPI, wal_bytes = dWB
+ FROM (SELECT relid, wal_records AS dWR, wal_fpi AS dFPI, wal_bytes AS dWB
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = iWR, wal_fpi = iFPI, wal_bytes = iWB
+ FROM (SELECT relid, wal_records AS iWR, wal_fpi AS iFPI, wal_bytes AS iWB
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+step s1_checkpoint: CHECKPOINT;
+step s1_vacuum: VACUUM vestat;
+step s1_checkpoint: CHECKPOINT;
+step s1_difference:
+ SELECT t1.wal_records - t0.wal_records > 0 AS dWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS dFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS dWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+ SELECT t1.wal_records - t0.wal_records > 0 AS iWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS iFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS iWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+dwr|dfpi|dwb
+---+----+---
+t |t |t
+(1 row)
+
+iwr|ifpi|iwb
+---+----+---
+t |t |t
+(1 row)
+
+step s1_save_walls:
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = dWR, wal_fpi = dFPI, wal_bytes = dWB
+ FROM (SELECT relid, wal_records AS dWR, wal_fpi AS dFPI, wal_bytes AS dWB
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = iWR, wal_fpi = iFPI, wal_bytes = iWB
+ FROM (SELECT relid, wal_records AS iWR, wal_fpi AS iFPI, wal_bytes AS iWB
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+step s1_delete_full_table: DELETE FROM vestat;
+step s1_checkpoint: CHECKPOINT;
+step s1_vacuum_parallel: VACUUM (PARALLEL 2, INDEX_CLEANUP ON) vestat;
+step s1_checkpoint: CHECKPOINT;
+step s1_difference:
+ SELECT t1.wal_records - t0.wal_records > 0 AS dWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS dFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS dWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+ SELECT t1.wal_records - t0.wal_records > 0 AS iWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS iFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS iWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+dwr|dfpi|dwb
+---+----+---
+t |t |t
+(1 row)
+
+iwr|ifpi|iwb
+---+----+---
+t |t |t
+(1 row)
+
+step s1_save_walls:
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = dWR, wal_fpi = dFPI, wal_bytes = dWB
+ FROM (SELECT relid, wal_records AS dWR, wal_fpi AS dFPI, wal_bytes AS dWB
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = iWR, wal_fpi = iFPI, wal_bytes = iWB
+ FROM (SELECT relid, wal_records AS iWR, wal_fpi AS iFPI, wal_bytes AS iWB
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+step s1_insert: INSERT INTO vestat(x) SELECT id FROM generate_series(1,770) As id;
+step s1_update: UPDATE vestat SET x = x+1;
+ERROR: duplicate key value violates unique constraint "vestat_pkey"
+step s1_checkpoint: CHECKPOINT;
+step s1_vacuum_full: VACUUM FULL vestat;
+step s1_checkpoint: CHECKPOINT;
+step s1_difference:
+ SELECT t1.wal_records - t0.wal_records > 0 AS dWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS dFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS dWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+ SELECT t1.wal_records - t0.wal_records > 0 AS iWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS iFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS iWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+dwr|dfpi|dwb
+---+----+---
+f |f |f
+(1 row)
+
+iwr|ifpi|iwb
+---+----+---
+f |f |f
+(1 row)
+
+step s1_save_walls:
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = dWR, wal_fpi = dFPI, wal_bytes = dWB
+ FROM (SELECT relid, wal_records AS dWR, wal_fpi AS dFPI, wal_bytes AS dWB
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = iWR, wal_fpi = iFPI, wal_bytes = iWB
+ FROM (SELECT relid, wal_records AS iWR, wal_fpi AS iFPI, wal_bytes AS iWB
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+step s1_checkpoint: CHECKPOINT;
+step s1_delete_full_table: DELETE FROM vestat;
+step s1_trancate: TRUNCATE vestat;
+step s1_vacuum: VACUUM vestat;
+step s1_checkpoint: CHECKPOINT;
+step s1_difference:
+ SELECT t1.wal_records - t0.wal_records > 0 AS dWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS dFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS dWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+ SELECT t1.wal_records - t0.wal_records > 0 AS iWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS iFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS iWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+dwr|dfpi|dwb
+---+----+---
+t |f |t
+(1 row)
+
+iwr|ifpi|iwb
+---+----+---
+t |f |t
+(1 row)
+
+step s1_save_walls:
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = dWR, wal_fpi = dFPI, wal_bytes = dWB
+ FROM (SELECT relid, wal_records AS dWR, wal_fpi AS dFPI, wal_bytes AS dWB
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = iWR, wal_fpi = iFPI, wal_bytes = iWB
+ FROM (SELECT relid, wal_records AS iWR, wal_fpi AS iFPI, wal_bytes AS iWB
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
diff --git a/src/test/isolation/expected/vacuum-extending.out b/src/test/isolation/expected/vacuum-extending.out
new file mode 100644
index 00000000000..6516d31be60
--- /dev/null
+++ b/src/test/isolation/expected/vacuum-extending.out
@@ -0,0 +1,68 @@
+Parsed test spec with 2 sessions
+
+starting permutation: s2_insert s2_print_vacuum_stats_table s1_begin_repeatable_read s2_update s2_insert_interrupt s2_vacuum s2_print_vacuum_stats_table s1_commit s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname|tuples_deleted|dead_tuples|tuples_frozen
+-------+--------------+-----------+-------------
+(0 rows)
+
+step s1_begin_repeatable_read:
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+
+count
+-----
+ 100
+(1 row)
+
+step s2_update: UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900;
+step s2_insert_interrupt: INSERT INTO test_vacuum_stat_isolation values (1,1);
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 0| 100| 0
+(1 row)
+
+step s1_commit: COMMIT;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 100| 101
+(1 row)
+
+
+starting permutation: s2_insert s2_delete s2_checkpoint s2_vacuum s2_print_vacuum_stats_table
+step s2_insert: INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival;
+step s2_delete: DELETE FROM test_vacuum_stat_isolation where id > 900;
+step s2_checkpoint: CHECKPOINT;
+step s2_vacuum: VACUUM test_vacuum_stat_isolation;
+step s2_print_vacuum_stats_table:
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+
+relname |tuples_deleted|dead_tuples|tuples_frozen
+--------------------------+--------------+-----------+-------------
+test_vacuum_stat_isolation| 100| 0| 222
+(1 row)
+
diff --git a/src/test/isolation/isolation_schedule b/src/test/isolation/isolation_schedule
index 0342eb39e40..13e92bfc220 100644
--- a/src/test/isolation/isolation_schedule
+++ b/src/test/isolation/isolation_schedule
@@ -92,6 +92,8 @@ test: timeouts
test: vacuum-concurrent-drop
test: vacuum-conflict
test: vacuum-skip-locked
+test: vacuum-extending
+test: vacuum-extended-statistic
test: stats
test: horizons
test: predicate-hash
diff --git a/src/test/isolation/specs/vacuum-extended-statistic.spec b/src/test/isolation/specs/vacuum-extended-statistic.spec
new file mode 100644
index 00000000000..f749be8b020
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extended-statistic.spec
@@ -0,0 +1,179 @@
+# A number of tests dedicated to verification of the 'Extended Vacuum Statistics'
+# feature.
+# By default, statistics has a volatile nature. So, selection result can depend
+# on a bunch of things. Here some trivial tests are performed that should work
+# in the most cases.
+# Test for checking pages: frozen, scanned, removed, number of tuple_deleted in pgpro_stats_vacuum_tables.
+# Besides, this test check pages scanned, pages removed, tuples_deleted in pgpro_stats_vacuum_tables and
+# wal values statistic collected over vacuum operation as for tables as for indexes.
+
+setup
+{
+ CREATE TABLE vestat (x int primary key) WITH (autovacuum_enabled = off);
+
+ CREATE TABLE vacuum_wal_stats_table
+ (relid int, wal_records int, wal_fpi int, wal_bytes int);
+ insert into vacuum_wal_stats_table (relid)
+ select oid from pg_class c
+ WHERE relname = 'vestat';
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = 0, wal_fpi = 0, wal_bytes = 0;
+
+ CREATE TABLE vacuum_wal_stats_index
+ (relid int, wal_records int, wal_fpi int, wal_bytes int);
+ insert into vacuum_wal_stats_index (relid)
+ select oid from pg_class c
+ WHERE relname = 'vestat_pkey';
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = 0, wal_fpi = 0, wal_bytes = 0;
+
+ SET track_io_timing = on;
+ SHOW track_counts; -- must be on
+ SET track_functions TO 'all';
+
+}
+
+teardown
+{
+ RESET vacuum_freeze_min_age;
+ RESET vacuum_freeze_table_age;
+ DROP TABLE vestat CASCADE;
+ DROP TABLE vacuum_wal_stats_index;
+ DROP TABLE vacuum_wal_stats_table;
+}
+
+session s1
+step s1_set_agressive_vacuum { SET vacuum_freeze_min_age = 0; }
+step s1_insert { INSERT INTO vestat(x) SELECT id FROM generate_series(1,770) As id; }
+step s1_update { UPDATE vestat SET x = x+1; }
+step s1_delete_half_table { DELETE FROM vestat WHERE x % 2 = 0; }
+step s1_delete_full_table { DELETE FROM vestat; }
+step s1_vacuum { VACUUM vestat; }
+step s1_vacuum_full { VACUUM FULL vestat; }
+step s1_vacuum_parallel { VACUUM (PARALLEL 2, INDEX_CLEANUP ON) vestat; }
+step s1_analyze { ANALYZE vestat; }
+step s1_trancate { TRUNCATE vestat; }
+step s1_checkpoint { CHECKPOINT; }
+step s1_print_vacuum_stats_tables
+{
+ SELECT vt.relname,
+ pages_frozen,
+ tuples_deleted,
+ pages_scanned,
+ pages_removed
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid;
+}
+
+step s1_print_vacuum_stats_indexes
+{
+ SELECT vt.relname,
+ pages_deleted,
+ tuples_deleted
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid;
+}
+
+step s1_save_walls
+{
+ UPDATE vacuum_wal_stats_table SET
+ wal_records = dWR, wal_fpi = dFPI, wal_bytes = dWB
+ FROM (SELECT relid, wal_records AS dWR, wal_fpi AS dFPI, wal_bytes AS dWB
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'vestat' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+
+ UPDATE vacuum_wal_stats_index SET
+ wal_records = iWR, wal_fpi = iFPI, wal_bytes = iWB
+ FROM (SELECT relid, wal_records AS iWR, wal_fpi AS iFPI, wal_bytes AS iWB
+ FROM pg_stats_vacuum_indexes vt, pg_class c
+ WHERE vt.relname = 'vestat_pkey' AND
+ vt.relid = c.oid) t
+ WHERE
+ t.relid = t.relid;
+}
+
+step s1_difference
+{
+ SELECT t1.wal_records - t0.wal_records > 0 AS dWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS dFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS dWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+
+ SELECT t1.wal_records - t0.wal_records > 0 AS iWR,
+ t1.wal_fpi - t0.wal_fpi > 0 AS iFPI,
+ t1.wal_bytes - t0.wal_bytes > 0 AS iWB
+ FROM vacuum_wal_stats_table t0, pg_stats_vacuum_tables t1
+ WHERE t0.relid = t1.relid;
+}
+
+permutation
+ s1_insert
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_set_agressive_vacuum
+ s1_analyze
+ s1_delete_half_table
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_checkpoint
+ s1_vacuum
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_delete_full_table
+ s1_checkpoint
+ s1_vacuum_parallel
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_insert
+ s1_update
+ s1_checkpoint
+ s1_vacuum_full
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+ s1_checkpoint
+ s1_delete_full_table
+ s1_trancate
+ s1_checkpoint
+ s1_vacuum
+ s1_print_vacuum_stats_tables
+ s1_print_vacuum_stats_indexes
+
+permutation
+ s1_insert
+ s1_set_agressive_vacuum
+ s1_analyze
+ s1_delete_half_table
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_checkpoint
+ s1_vacuum
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_delete_full_table
+ s1_checkpoint
+ s1_vacuum_parallel
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_insert
+ s1_update
+ s1_checkpoint
+ s1_vacuum_full
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
+ s1_checkpoint
+ s1_delete_full_table
+ s1_trancate
+ s1_vacuum
+ s1_checkpoint
+ s1_difference
+ s1_save_walls
\ No newline at end of file
diff --git a/src/test/isolation/specs/vacuum-extending.spec b/src/test/isolation/specs/vacuum-extending.spec
new file mode 100644
index 00000000000..9b4dac68a3c
--- /dev/null
+++ b/src/test/isolation/specs/vacuum-extending.spec
@@ -0,0 +1,58 @@
+# Test for checking dead_tuples, tuples_deleted and frozen tuples in pg_stats_vacuum_tables.
+# Dead_tuples values are counted when vacuum cannot clean up unused tuples while lock is using another transaction.
+# Dead_tuples aren't increased after releasing lock compared with tuples_deleted, which increased
+# by the value of the cleared tuples that the vacuum managed to clear.
+
+setup
+{
+ CREATE TABLE test_vacuum_stat_isolation(id int, ival int) WITH (autovacuum_enabled = off);
+ SET track_io_timing = on;
+}
+
+teardown
+{
+ DROP TABLE test_vacuum_stat_isolation CASCADE;
+ RESET track_io_timing;
+}
+
+session s1
+step s1_begin_repeatable_read {
+ BEGIN transaction ISOLATION LEVEL REPEATABLE READ;
+ select count(ival) from test_vacuum_stat_isolation where id>900;
+ }
+step s1_commit { COMMIT; }
+
+session s2
+step s2_insert { INSERT INTO test_vacuum_stat_isolation(id, ival) SELECT ival, ival%10 FROM generate_series(1,1000) As ival; }
+step s2_update { UPDATE test_vacuum_stat_isolation SET ival = ival + 2 where id > 900; }
+step s2_delete { DELETE FROM test_vacuum_stat_isolation where id > 900; }
+step s2_insert_interrupt { INSERT INTO test_vacuum_stat_isolation values (1,1); }
+step s2_vacuum { VACUUM test_vacuum_stat_isolation; }
+step s2_checkpoint { CHECKPOINT; }
+step s2_print_vacuum_stats_table
+{
+ SELECT
+ vt.relname, vt.tuples_deleted, vt.dead_tuples, vt.tuples_frozen
+ FROM pg_stats_vacuum_tables vt, pg_class c
+ WHERE vt.relname = 'test_vacuum_stat_isolation' AND vt.relid = c.oid;
+}
+
+permutation
+ s2_insert
+ s2_print_vacuum_stats_table
+ s1_begin_repeatable_read
+ s2_update
+ s2_insert_interrupt
+ s2_vacuum
+ s2_print_vacuum_stats_table
+ s1_commit
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
+
+permutation
+ s2_insert
+ s2_delete
+ s2_checkpoint
+ s2_vacuum
+ s2_print_vacuum_stats_table
\ No newline at end of file
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 9d047b21b88..8ba58f49f4c 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -32,9 +32,12 @@ WHERE p1.prolang = 0 OR p1.prorettype = 0 OR
prokind NOT IN ('f', 'a', 'w', 'p') OR
provolatile NOT IN ('i', 's', 'v') OR
proparallel NOT IN ('s', 'r', 'u');
- oid | proname
------+---------
-(0 rows)
+ oid | proname
+------+--------------------------
+ 4701 | pg_stats_vacuum_tables
+ 4702 | pg_stats_vacuum_indexes
+ 4703 | pg_stats_vacuum_database
+(3 rows)
-- prosrc should never be null; it can be empty only if prosqlbody isn't null
SELECT p1.oid, p1.proname
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index ef658ad7405..56a76f45693 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2606,6 +2606,85 @@ pg_stats_ext_exprs| SELECT cn.nspname AS schemaname,
JOIN LATERAL ( SELECT unnest(pg_get_statisticsobjdef_expressions(s.oid)) AS expr,
unnest(sd.stxdexpr) AS a) stat ON ((stat.expr IS NOT NULL)))
WHERE (pg_has_role(c.relowner, 'USAGE'::text) AND ((c.relrowsecurity = false) OR (NOT row_security_active(c.oid))));
+pg_stats_vacuum_database| SELECT db.oid AS dboid,
+ stats.db_blks_read,
+ stats.db_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_namespace ns,
+ LATERAL pg_stats_vacuum_database(db.oid) stats(dboid, db_blks_read, db_blks_hit, total_blks_dirtied, total_blks_written, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE (db.datname = current_database());
+pg_stats_vacuum_indexes| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_deleted,
+ stats.tuples_deleted,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stats_vacuum_indexes(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_deleted, tuples_deleted, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
+pg_stats_vacuum_tables| SELECT rel.oid AS relid,
+ ns.nspname AS schema,
+ rel.relname,
+ stats.total_blks_read,
+ stats.total_blks_hit,
+ stats.total_blks_dirtied,
+ stats.total_blks_written,
+ stats.rel_blks_read,
+ stats.rel_blks_hit,
+ stats.pages_scanned,
+ stats.pages_removed,
+ stats.pages_frozen,
+ stats.pages_all_visible,
+ stats.tuples_deleted,
+ stats.tuples_frozen,
+ stats.dead_tuples,
+ stats.index_vacuum_count,
+ stats.rev_all_frozen_pages,
+ stats.rev_all_visible_pages,
+ stats.wal_records,
+ stats.wal_fpi,
+ stats.wal_bytes,
+ stats.blk_read_time,
+ stats.blk_write_time,
+ stats.delay_time,
+ stats.system_time,
+ stats.user_time,
+ stats.total_time,
+ stats.interrupts
+ FROM pg_database db,
+ pg_class rel,
+ pg_namespace ns,
+ LATERAL pg_stats_vacuum_tables(db.oid, rel.oid) stats(relid, total_blks_read, total_blks_hit, total_blks_dirtied, total_blks_written, rel_blks_read, rel_blks_hit, pages_scanned, pages_removed, pages_frozen, pages_all_visible, tuples_deleted, tuples_frozen, dead_tuples, index_vacuum_count, rev_all_frozen_pages, rev_all_visible_pages, wal_records, wal_fpi, wal_bytes, blk_read_time, blk_write_time, delay_time, system_time, user_time, total_time, interrupts)
+ WHERE ((db.datname = current_database()) AND (rel.oid = stats.relid) AND (ns.oid = rel.relnamespace));
pg_tables| SELECT n.nspname AS schemaname,
c.relname AS tablename,
pg_get_userbyid(c.relowner) AS tableowner,
--
2.34.1
