On Mon, 2022-04-04 at 11:15 +0530, Bharath Rupireddy wrote: > 1) Why can't rmid be chosen by the extensions in sequential order > from > (129 till 255), say, to start with a columnar extension can choose > 129, another extension can register 130 and so on right?
I'm not sure what you mean by "chosen by the extensions in sequential order". If you mean: (a) that it would depend on the library load order, and that postgres would assign the ID; then that won't work because the rmgr IDs need to be stable across restarts and config changes (b) that there should be a convention where people reserve IDs on the wiki page in sequential order; then that's fine with me > 2) RM_MAX_ID is now UINT8_MAX - do we need to make it configurable? > or > do we think that only a few (127) custom rmgrs can exist? 127 should be plenty for quite a long time. If we need more, we can come up with a different solution (which is OK because the WAL format changes in new major versions). > 3) Do we need to talk about extensible rmgrs and > https://wiki.postgresql.org/wiki/ExtensibleRmgr in documentation > somewhere? This will help extension developers to refer when needed. We don't have user-facing documentation for every extension hook, and I don't think it would be a good idea to document this one (at least in the user-facing docs). Otherwise, we'd have to document the whole resource manager interface, and that seems like way too much of the internals. > 4) Do we need to change this description? > <para> > The default value of this setting is the empty string, which > disables > the feature. It can be set to <literal>all</literal> to > check all > records, or to a comma-separated list of resource managers to > check > only records originating from those resource > managers. Currently, > the supported resource managers are <literal>heap</literal>, > <literal>heap2</literal>, <literal>btree</literal>, > <literal>hash</literal>, > <literal>gin</literal>, <literal>gist</literal>, > <literal>sequence</literal>, > <literal>spgist</literal>, <literal>brin</literal>, and > <literal>generic</literal>. Onl > superusers can change this setting. Yes, you're right, I updated the docs for pg_waldump and the wal_consistency_checking GUC. > 5) Do we need to add a PGDLLIMPORT qualifier to RegisterCustomRmgr() > (possibly for RmgrStartup, RmgrTable, RmgrCleanup as well?) so that > the Windows versions of extensions can use this feature? That seems to only be required for variables, not functions. I don't think we want to mark RmgrTable this way, because it's not intended to be accessed by extensions directly. It's accessed by GetRmgr(), which is 'static inline', but that's also not intended to be used by extensions. > 6) Should the below strcmp pg_strcmp? The custom rmgrs can still use > rm_name with different cases and below code fail to catch it? > + if (!strcmp(existing_rmgr->rm_name, rmgr->rm_name)) > + ereport(PANIC, > + (errmsg("custom rmgr \"%s\" has the same name as builtin rmgr", > + existing_rmgr->rm_name))); There are already a lot of places where users can choose identifiers that differ only in uppercase/lowercase. I don't see a reason to make an exception here. > 7) What's the intention of the below code? It seems like below it > checks if there's already a rmgr with the given name (RM_MAX_ID). Had > it been RM_MAX_BUILTIN_ID instead of RM_MAX_ID, the error message > does make sense. Is the intention here to not have duplicate rmgrs in > the entire RM_MAX_ID(both builtin and custom)? Thank you. I redid a lot of the error messages and checked them out to make sure they are helpful. > 8) Per https://wiki.postgresql.org/wiki/ExtensibleRmgr, 128 is > reserved, can we have it as a macro? Or something like > (RM_MAX_ID/2+1) It's already defined as RM_EXPERIMENTAL_ID. Is that what you meant or did I misunderstand? I don't see the point in defining it as RM_MAX_ID/2+1. > 9) Thinking if there's a way to test the code, maybe exposing > RegisterCustomRmgr as a function? I think we need to have a dummy > extension that will be used for testing this kind of patches/features > but that's a separate discussion IMO. It's already exposed as a C function in xlog_internal.h. You mean as a SQL function? I don't think that makes sense. It can only be called while processing shared_preload_libraries (on server startup) anyway. Other changes this version: * implemented Andres's proposed performance change[1] * fix wal_consistency_checking * general cleanup Regards, Jeff Davis [1] https://postgr.es/m/20220404043337.ocjnni7hknjsi...@alap3.anarazel.de
From 7f6acaf93d279baa464ae3ba2be173a01d969402 Mon Sep 17 00:00:00 2001 From: Jeff Davis <j...@j-davis.com> Date: Sat, 6 Nov 2021 13:01:38 -0700 Subject: [PATCH] Extensible rmgr. Allow extensions to specify a new custom rmgr, which allows specialized WAL. This is meant to be used by a custom Table Access Method, which would not otherwise be able to offer logical decoding/replication. It may also be used by new Index Access Methods. Prior to this commit, only Generic WAL was available, which offers support for recovery and physical replication but not logical replication. Reviewed-by: Julien Rouhaud, Andres Freund Discussion: https://postgr.es/m/ed1fb2e22d15d3563ae0eb610f7b61bb15999c0a.camel%40j-davis.com --- doc/src/sgml/config.sgml | 9 +++ doc/src/sgml/ref/pg_waldump.sgml | 8 ++ src/backend/access/transam/rmgr.c | 91 ++++++++++++++++++++++- src/backend/access/transam/xlogreader.c | 2 +- src/backend/access/transam/xlogrecovery.c | 33 +++----- src/backend/replication/logical/decode.c | 8 +- src/backend/utils/misc/guc.c | 26 +++++-- src/bin/pg_rewind/parsexlog.c | 11 ++- src/bin/pg_waldump/pg_waldump.c | 66 +++++++++++----- src/bin/pg_waldump/rmgrdesc.c | 67 ++++++++++++++++- src/bin/pg_waldump/rmgrdesc.h | 2 +- src/include/access/rmgr.h | 15 +++- src/include/access/xlog_internal.h | 23 +++++- 13 files changed, 300 insertions(+), 61 deletions(-) diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml index 43e4ade83e0..c80327af67a 100644 --- a/doc/src/sgml/config.sgml +++ b/doc/src/sgml/config.sgml @@ -11148,6 +11148,15 @@ LOG: CleanUpLock: deleting: lock(0xb7acd844) id(24688,24696,0,0,0,1) <literal>spgist</literal>, <literal>brin</literal>, and <literal>generic</literal>. Only superusers can change this setting. </para> + + <para> + Extensions may define additional resource managers with modules loaded + in <xref linkend="guc-shared-preload-libraries"/>. However, the names + of the custom resource managers are not available at the time the + configuration file is processed, so they must be referred to + numerically with the form <literal>custom###</literal>, where + "<literal>###</literal>" is the three-digit resource manager ID. + </para> </listitem> </varlistentry> diff --git a/doc/src/sgml/ref/pg_waldump.sgml b/doc/src/sgml/ref/pg_waldump.sgml index 1a05af5d972..57746d9421f 100644 --- a/doc/src/sgml/ref/pg_waldump.sgml +++ b/doc/src/sgml/ref/pg_waldump.sgml @@ -173,6 +173,14 @@ PostgreSQL documentation If <literal>list</literal> is passed as name, print a list of valid resource manager names, and exit. </para> + <para> + Extensions may define custom resource managers, but pg_waldump does + not load the extension module and therefore does not recognize custom + resource managers by name. Instead, you can specify the custom + resource managers as <literal>custom###</literal> where + "<literal>###</literal>" is the three-digit resource manager ID. Names + of this form will always be considered valid. + </para> </listitem> </varlistentry> diff --git a/src/backend/access/transam/rmgr.c b/src/backend/access/transam/rmgr.c index f8847d5aebf..d215116a912 100644 --- a/src/backend/access/transam/rmgr.c +++ b/src/backend/access/transam/rmgr.c @@ -24,6 +24,7 @@ #include "commands/dbcommands_xlog.h" #include "commands/sequence.h" #include "commands/tablespace.h" +#include "miscadmin.h" #include "replication/decode.h" #include "replication/message.h" #include "replication/origin.h" @@ -34,6 +35,94 @@ #define PG_RMGR(symname,name,redo,desc,identify,startup,cleanup,mask,decode) \ { name, redo, desc, identify, startup, cleanup, mask, decode }, -const RmgrData RmgrTable[RM_MAX_ID + 1] = { +RmgrData RmgrTable[RM_MAX_ID + 1] = { #include "access/rmgrlist.h" }; + +/* + * Start up all resource managers. + */ +void +RmgrStartup(void) +{ + for (int rmid = 0; rmid <= RM_MAX_ID; rmid++) + { + if (!RmgrExists(rmid)) + continue; + + if (RmgrTable[rmid].rm_startup != NULL) + RmgrTable[rmid].rm_startup(); + } +} + +/* + * Clean up all resource managers. + */ +void +RmgrCleanup(void) +{ + for (int rmid = 0; rmid <= RM_MAX_ID; rmid++) + { + if (!RmgrExists(rmid)) + continue; + + if (RmgrTable[rmid].rm_cleanup != NULL) + RmgrTable[rmid].rm_cleanup(); + } +} + +/* + * Emit PANIC message when we encounter a record with an RmgrId we don't recognize. + */ +void +RmgrNotFound(RmgrId rmid) +{ + ereport(PANIC, (errmsg("resource manager with ID %d not registered", rmid), + errhint("Include the extension module that implements this resource manager in shared_preload_libraries."))); +} + +/* + * Register a new custom rmgr. + * + * Refer to https://wiki.postgresql.org/wiki/ExtensibleRmgr to reserve a + * unique RmgrId for your extension, to avoid conflicts. During development, + * use RM_EXPERIMENTAL_ID. + */ +void +RegisterCustomRmgr(RmgrId rmid, RmgrData *rmgr) +{ + if (rmgr->rm_name == NULL) + ereport(PANIC, errmsg("custom resource manager is invalid")); + + if (rmid < RM_MIN_CUSTOM_ID) + ereport(PANIC, errmsg("custom resource manager ID %d is out of range", rmid)); + + if (!process_shared_preload_libraries_in_progress) + ereport(PANIC, + (errmsg("failed to register custom resource manager \"%s\" with ID %d", rmgr->rm_name, rmid), + errdetail("Custom resource manager must be registered while initializing modules in shared_preload_libraries."))); + + if (RmgrTable[rmid].rm_name != NULL) + ereport(PANIC, + (errmsg("failed to register custom resource manager \"%s\" with ID %d", rmgr->rm_name, rmid), + errdetail("Custom resource manager \"%s\" already registered with the same ID.", + RmgrTable[rmid].rm_name))); + + /* check for existing rmgr with the same name */ + for (int i = 0; i <= RM_MAX_ID; i++) + { + if (!RmgrExists(i)) + continue; + + if (!strcmp(RmgrTable[i].rm_name, rmgr->rm_name)) + ereport(PANIC, + (errmsg("failed to register custom resource manager \"%s\" with ID %d", rmgr->rm_name, rmid), + errdetail("Existing resource manager with ID %d has the same name.", i))); + } + + /* register it */ + RmgrTable[rmid] = *rmgr; + ereport(LOG, + (errmsg("registered custom resource manager \"%s\" with ID %d", + rmgr->rm_name, rmid))); +} diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c index e437c429920..161cf13fed2 100644 --- a/src/backend/access/transam/xlogreader.c +++ b/src/backend/access/transam/xlogreader.c @@ -1102,7 +1102,7 @@ ValidXLogRecordHeader(XLogReaderState *state, XLogRecPtr RecPtr, (uint32) SizeOfXLogRecord, record->xl_tot_len); return false; } - if (record->xl_rmid > RM_MAX_ID) + if (!RMID_IS_VALID(record->xl_rmid)) { report_invalid_record(state, "invalid resource manager ID %u at %X/%X", diff --git a/src/backend/access/transam/xlogrecovery.c b/src/backend/access/transam/xlogrecovery.c index 8d2395dae25..999e24fcf2c 100644 --- a/src/backend/access/transam/xlogrecovery.c +++ b/src/backend/access/transam/xlogrecovery.c @@ -1531,7 +1531,6 @@ ShutdownWalRecovery(void) void PerformWalRecovery(void) { - int rmid; XLogRecord *record; bool reachedRecoveryTarget = false; TimeLineID replayTLI; @@ -1604,12 +1603,7 @@ PerformWalRecovery(void) InRedo = true; - /* Initialize resource managers */ - for (rmid = 0; rmid <= RM_MAX_ID; rmid++) - { - if (RmgrTable[rmid].rm_startup != NULL) - RmgrTable[rmid].rm_startup(); - } + RmgrStartup(); ereport(LOG, (errmsg("redo starts at %X/%X", @@ -1746,12 +1740,7 @@ PerformWalRecovery(void) } } - /* Allow resource managers to do any required cleanup. */ - for (rmid = 0; rmid <= RM_MAX_ID; rmid++) - { - if (RmgrTable[rmid].rm_cleanup != NULL) - RmgrTable[rmid].rm_cleanup(); - } + RmgrCleanup(); ereport(LOG, (errmsg("redo done at %X/%X system usage: %s", @@ -1871,7 +1860,7 @@ ApplyWalRecord(XLogReaderState *xlogreader, XLogRecord *record, TimeLineID *repl xlogrecovery_redo(xlogreader, *replayTLI); /* Now apply the WAL record itself */ - RmgrTable[record->xl_rmid].rm_redo(xlogreader); + GetRmgr(record->xl_rmid).rm_redo(xlogreader); /* * After redo, check whether the backup pages associated with the WAL @@ -2101,20 +2090,20 @@ rm_redo_error_callback(void *arg) void xlog_outdesc(StringInfo buf, XLogReaderState *record) { - RmgrId rmid = XLogRecGetRmid(record); + RmgrData rmgr = GetRmgr(XLogRecGetRmid(record)); uint8 info = XLogRecGetInfo(record); const char *id; - appendStringInfoString(buf, RmgrTable[rmid].rm_name); + appendStringInfoString(buf, rmgr.rm_name); appendStringInfoChar(buf, '/'); - id = RmgrTable[rmid].rm_identify(info); + id = rmgr.rm_identify(info); if (id == NULL) appendStringInfo(buf, "UNKNOWN (%X): ", info & ~XLR_INFO_MASK); else appendStringInfo(buf, "%s: ", id); - RmgrTable[rmid].rm_desc(buf, record); + rmgr.rm_desc(buf, record); } #ifdef WAL_DEBUG @@ -2263,7 +2252,7 @@ getRecordTimestamp(XLogReaderState *record, TimestampTz *recordXtime) static void verifyBackupPageConsistency(XLogReaderState *record) { - RmgrId rmid = XLogRecGetRmid(record); + RmgrData rmgr = GetRmgr(XLogRecGetRmid(record)); RelFileNode rnode; ForkNumber forknum; BlockNumber blkno; @@ -2343,10 +2332,10 @@ verifyBackupPageConsistency(XLogReaderState *record) * If masking function is defined, mask both the primary and replay * images */ - if (RmgrTable[rmid].rm_mask != NULL) + if (rmgr.rm_mask != NULL) { - RmgrTable[rmid].rm_mask(replay_image_masked, blkno); - RmgrTable[rmid].rm_mask(primary_image_masked, blkno); + rmgr.rm_mask(replay_image_masked, blkno); + rmgr.rm_mask(primary_image_masked, blkno); } /* Time to compare the primary and replay images. */ diff --git a/src/backend/replication/logical/decode.c b/src/backend/replication/logical/decode.c index 77bc7aea7a0..c6ea7c98e15 100644 --- a/src/backend/replication/logical/decode.c +++ b/src/backend/replication/logical/decode.c @@ -94,7 +94,7 @@ LogicalDecodingProcessRecord(LogicalDecodingContext *ctx, XLogReaderState *recor { XLogRecordBuffer buf; TransactionId txid; - RmgrId rmid; + RmgrData rmgr; buf.origptr = ctx->reader->ReadRecPtr; buf.endptr = ctx->reader->EndRecPtr; @@ -115,10 +115,10 @@ LogicalDecodingProcessRecord(LogicalDecodingContext *ctx, XLogReaderState *recor buf.origptr); } - rmid = XLogRecGetRmid(record); + rmgr = GetRmgr(XLogRecGetRmid(record)); - if (RmgrTable[rmid].rm_decode != NULL) - RmgrTable[rmid].rm_decode(ctx, &buf); + if (rmgr.rm_decode != NULL) + rmgr.rm_decode(ctx, &buf); else { /* just deal with xid, and done */ diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index 9e8ab1420d9..5a08769e497 100644 --- a/src/backend/utils/misc/guc.c +++ b/src/backend/utils/misc/guc.c @@ -11769,26 +11769,40 @@ check_wal_consistency_checking(char **newval, void **extra, GucSource source) { char *tok = (char *) lfirst(l); bool found = false; - RmgrId rmid; + int rmid; /* Check for 'all'. */ if (pg_strcasecmp(tok, "all") == 0) { - for (rmid = 0; rmid <= RM_MAX_ID; rmid++) - if (RmgrTable[rmid].rm_mask != NULL) + for (rmid = 0; rmid <= RM_MAX_BUILTIN_ID; rmid++) + if (GetRmgr(rmid).rm_mask != NULL) newwalconsistency[rmid] = true; found = true; } + else if (sscanf(tok, "custom%03d", &rmid) == 1) + { + /* + * The server hasn't loaded the shared_preload_libraries yet, so + * we don't have access to the custom resource manager + * names. Allow the form "custom###", where "###" is the 3-digit + * resource manager ID. + */ + if (RMID_IS_CUSTOM(rmid)) + { + newwalconsistency[rmid] = true; + found = true; + } + } else { /* * Check if the token matches with any individual resource * manager. */ - for (rmid = 0; rmid <= RM_MAX_ID; rmid++) + for (rmid = 0; rmid <= RM_MAX_BUILTIN_ID; rmid++) { - if (pg_strcasecmp(tok, RmgrTable[rmid].rm_name) == 0 && - RmgrTable[rmid].rm_mask != NULL) + if (GetRmgr(rmid).rm_mask != NULL && + pg_strcasecmp(tok, GetRmgr(rmid).rm_name) == 0) { newwalconsistency[rmid] = true; found = true; diff --git a/src/bin/pg_rewind/parsexlog.c b/src/bin/pg_rewind/parsexlog.c index 49966e7b7fd..dfa836d1561 100644 --- a/src/bin/pg_rewind/parsexlog.c +++ b/src/bin/pg_rewind/parsexlog.c @@ -25,8 +25,8 @@ #include "pg_rewind.h" /* - * RmgrNames is an array of resource manager names, to make error messages - * a bit nicer. + * RmgrNames is an array of the built-in resource manager names, to make error + * messages a bit nicer. */ #define PG_RMGR(symname,name,redo,desc,identify,startup,cleanup,mask,decode) \ name, @@ -35,6 +35,9 @@ static const char *RmgrNames[RM_MAX_ID + 1] = { #include "access/rmgrlist.h" }; +#define RmgrName(rmid) (((rmid) <= RM_MAX_BUILTIN_ID) ? \ + RmgrNames[rmid] : "custom") + static void extractPageInfo(XLogReaderState *record); static int xlogreadfd = -1; @@ -436,9 +439,9 @@ extractPageInfo(XLogReaderState *record) * track that change. */ pg_fatal("WAL record modifies a relation, but record type is not recognized: " - "lsn: %X/%X, rmgr: %s, info: %02X", + "lsn: %X/%X, rmid: %d, rmgr: %s, info: %02X", LSN_FORMAT_ARGS(record->ReadRecPtr), - RmgrNames[rmid], info); + rmid, RmgrName(rmid), info); } for (block_id = 0; block_id <= XLogRecMaxBlockId(record); block_id++) diff --git a/src/bin/pg_waldump/pg_waldump.c b/src/bin/pg_waldump/pg_waldump.c index 4cb40d068a9..4f47449a6cb 100644 --- a/src/bin/pg_waldump/pg_waldump.c +++ b/src/bin/pg_waldump/pg_waldump.c @@ -80,8 +80,8 @@ typedef struct XLogDumpStats uint64 count; XLogRecPtr startptr; XLogRecPtr endptr; - Stats rmgr_stats[RM_NEXT_ID]; - Stats record_stats[RM_NEXT_ID][MAX_XLINFO_TYPES]; + Stats rmgr_stats[RM_MAX_ID + 1]; + Stats record_stats[RM_MAX_ID + 1][MAX_XLINFO_TYPES]; } XLogDumpStats; #define fatal_error(...) do { pg_log_fatal(__VA_ARGS__); exit(EXIT_FAILURE); } while(0) @@ -104,9 +104,9 @@ print_rmgr_list(void) { int i; - for (i = 0; i <= RM_MAX_ID; i++) + for (i = 0; i <= RM_MAX_BUILTIN_ID; i++) { - printf("%s\n", RmgrDescTable[i].rm_name); + printf("%s\n", GetRmgrDesc(i)->rm_name); } } @@ -535,7 +535,7 @@ static void XLogDumpDisplayRecord(XLogDumpConfig *config, XLogReaderState *record) { const char *id; - const RmgrDescData *desc = &RmgrDescTable[XLogRecGetRmid(record)]; + const RmgrDescData *desc = GetRmgrDesc(XLogRecGetRmid(record)); uint32 rec_len; uint32 fpi_len; RelFileNode rnode; @@ -720,7 +720,7 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogDumpStats *stats) * calculate column totals. */ - for (ri = 0; ri < RM_NEXT_ID; ri++) + for (ri = 0; ri < RM_MAX_ID; ri++) { total_count += stats->rmgr_stats[ri].count; total_rec_len += stats->rmgr_stats[ri].rec_len; @@ -741,13 +741,18 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogDumpStats *stats) "Type", "N", "(%)", "Record size", "(%)", "FPI size", "(%)", "Combined size", "(%)", "----", "-", "---", "-----------", "---", "--------", "---", "-------------", "---"); - for (ri = 0; ri < RM_NEXT_ID; ri++) + for (ri = 0; ri <= RM_MAX_ID; ri++) { uint64 count, rec_len, fpi_len, tot_len; - const RmgrDescData *desc = &RmgrDescTable[ri]; + const RmgrDescData *desc; + + if (!RMID_IS_VALID(ri)) + continue; + + desc = GetRmgrDesc(ri); if (!config->stats_per_record) { @@ -756,6 +761,9 @@ XLogDumpDisplayStats(XLogDumpConfig *config, XLogDumpStats *stats) fpi_len = stats->rmgr_stats[ri].fpi_len; tot_len = rec_len + fpi_len; + if (RMID_IS_CUSTOM(ri) && count == 0) + continue; + XLogDumpStatsRow(desc->rm_name, count, total_count, rec_len, total_rec_len, fpi_len, total_fpi_len, tot_len, total_len); @@ -1000,7 +1008,7 @@ main(int argc, char **argv) break; case 'r': { - int i; + int rmid; if (pg_strcasecmp(optarg, "list") == 0) { @@ -1008,20 +1016,42 @@ main(int argc, char **argv) exit(EXIT_SUCCESS); } - for (i = 0; i <= RM_MAX_ID; i++) + /* + * First look for the generated name of a custom rmgr, of + * the form "custom###". We accept this form, because the + * custom rmgr module is not loaded, so there's no way to + * know the real name. This convention should be + * consistent with that in rmgrdesc.c. + */ + if (sscanf(optarg, "custom%03d", &rmid) == 1) { - if (pg_strcasecmp(optarg, RmgrDescTable[i].rm_name) == 0) + if (!RMID_IS_CUSTOM(rmid)) { - config.filter_by_rmgr[i] = true; - config.filter_by_rmgr_enabled = true; - break; + pg_log_error("custom resource manager \"%s\" does not exist", + optarg); + goto bad_argument; } + config.filter_by_rmgr[rmid] = true; + config.filter_by_rmgr_enabled = true; } - if (i > RM_MAX_ID) + else { - pg_log_error("resource manager \"%s\" does not exist", - optarg); - goto bad_argument; + /* then look for builtin rmgrs */ + for (rmid = 0; rmid <= RM_MAX_BUILTIN_ID; rmid++) + { + if (pg_strcasecmp(optarg, GetRmgrDesc(rmid)->rm_name) == 0) + { + config.filter_by_rmgr[rmid] = true; + config.filter_by_rmgr_enabled = true; + break; + } + } + if (rmid > RM_MAX_BUILTIN_ID) + { + pg_log_error("resource manager \"%s\" does not exist", + optarg); + goto bad_argument; + } } } break; diff --git a/src/bin/pg_waldump/rmgrdesc.c b/src/bin/pg_waldump/rmgrdesc.c index 6a4ebd1310b..7871a16b08a 100644 --- a/src/bin/pg_waldump/rmgrdesc.c +++ b/src/bin/pg_waldump/rmgrdesc.c @@ -35,6 +35,71 @@ #define PG_RMGR(symname,name,redo,desc,identify,startup,cleanup,mask,decode) \ { name, desc, identify}, -const RmgrDescData RmgrDescTable[RM_MAX_ID + 1] = { +static const RmgrDescData RmgrDescTable[RM_MAX_BUILTIN_ID + 1] = { #include "access/rmgrlist.h" }; + +/* + * We are unable to get the real name of a custom rmgr because the module is + * not loaded. Generate a table of numeric names of the form "custom###" where + * "###" is the 3-digit resource manager ID. + */ +#define CUSTOM_NUMERIC_NAME_LEN sizeof("custom###") + +static char CustomNumericNames[RM_N_CUSTOM_IDS][CUSTOM_NUMERIC_NAME_LEN] = {0}; +static RmgrDescData CustomRmgrDesc[RM_N_CUSTOM_IDS] = {0}; +static bool CustomRmgrDescInitialized = false; + +/* + * No information on custom resource managers; just print the ID. + */ +static void +default_desc(StringInfo buf, XLogReaderState *record) +{ + appendStringInfo(buf, "rmid: %d", XLogRecGetRmid(record)); +} + +/* + * No information on custom resource managers; just return NULL and let the + * caller handle it. + */ +static const char * +default_identify(uint8 info) +{ + return NULL; +} + +/* + * pg_waldump does not load the modules that register the resource managers; + * therefore we are missing the RmgrDesc information. Generate phony data + * where the name follows the convention "custom###" where ### is the 3-digit + * rmgr ID. + */ +static void +initialize_custom_rmgrs(void) +{ + for (int i = 0; i < RM_N_CUSTOM_IDS; i++) + { + snprintf(CustomNumericNames[i], CUSTOM_NUMERIC_NAME_LEN, + "custom%03d", i + RM_MIN_CUSTOM_ID); + CustomRmgrDesc[i].rm_name = CustomNumericNames[i]; + CustomRmgrDesc[i].rm_desc = default_desc; + CustomRmgrDesc[i].rm_identify = default_identify; + } + CustomRmgrDescInitialized = true; +} + +const RmgrDescData * +GetRmgrDesc(RmgrId rmid) +{ + Assert(RMID_IS_VALID(rmid)); + + if (RMID_IS_BUILTIN(rmid)) + return &RmgrDescTable[rmid]; + else + { + if (!CustomRmgrDescInitialized) + initialize_custom_rmgrs(); + return &CustomRmgrDesc[rmid - RM_MIN_CUSTOM_ID]; + } +} diff --git a/src/bin/pg_waldump/rmgrdesc.h b/src/bin/pg_waldump/rmgrdesc.h index 42f8483b482..f733cd467d5 100644 --- a/src/bin/pg_waldump/rmgrdesc.h +++ b/src/bin/pg_waldump/rmgrdesc.h @@ -18,6 +18,6 @@ typedef struct RmgrDescData const char *(*rm_identify) (uint8 info); } RmgrDescData; -extern const RmgrDescData RmgrDescTable[]; +extern const RmgrDescData *GetRmgrDesc(RmgrId rmid); #endif /* RMGRDESC_H */ diff --git a/src/include/access/rmgr.h b/src/include/access/rmgr.h index d9b512630ca..dfa8521997c 100644 --- a/src/include/access/rmgr.h +++ b/src/include/access/rmgr.h @@ -30,6 +30,19 @@ typedef enum RmgrIds #undef PG_RMGR -#define RM_MAX_ID (RM_NEXT_ID - 1) +#define RM_MAX_ID UINT8_MAX +#define RM_MAX_BUILTIN_ID (RM_NEXT_ID - 1) +#define RM_MIN_CUSTOM_ID 128 +#define RM_N_CUSTOM_IDS (RM_MAX_ID - RM_MIN_CUSTOM_ID + 1) +#define RMID_IS_BUILTIN(rmid) ((rmid) <= RM_MAX_BUILTIN_ID) +#define RMID_IS_CUSTOM(rmid) ((rmid) >= RM_MIN_CUSTOM_ID && (rmid) < RM_MAX_ID) +#define RMID_IS_VALID(rmid) (RMID_IS_BUILTIN((rmid)) || RMID_IS_CUSTOM((rmid))) + +/* + * RmgrId to use for extensions that require an RmgrId, but are still in + * development and have not reserved their own unique RmgrId yet. See: + * https://wiki.postgresql.org/wiki/ExtensibleRmgr + */ +#define RM_EXPERIMENTAL_ID 128 #endif /* RMGR_H */ diff --git a/src/include/access/xlog_internal.h b/src/include/access/xlog_internal.h index 0e94833129a..9370f346c83 100644 --- a/src/include/access/xlog_internal.h +++ b/src/include/access/xlog_internal.h @@ -304,7 +304,8 @@ struct XLogRecordBuffer; * rm_mask takes as input a page modified by the resource manager and masks * out bits that shouldn't be flagged by wal_consistency_checking. * - * RmgrTable[] is indexed by RmgrId values (see rmgrlist.h). + * RmgrTable[] is indexed by RmgrId values (see rmgrlist.h). If rm_name is + * NULL, the structure is considered invalid. */ typedef struct RmgrData { @@ -319,7 +320,25 @@ typedef struct RmgrData struct XLogRecordBuffer *buf); } RmgrData; -extern const RmgrData RmgrTable[]; +extern RmgrData RmgrTable[]; +extern void RmgrStartup(void); +extern void RmgrCleanup(void); +extern void RmgrNotFound(RmgrId rmid); +extern void RegisterCustomRmgr(RmgrId rmid, RmgrData *rmgr); + +static inline bool +RmgrExists(RmgrId rmid) +{ + return RmgrTable[rmid].rm_name != NULL; +} + +static inline RmgrData +GetRmgr(RmgrId rmid) +{ + if (unlikely(!RmgrExists(rmid))) + RmgrNotFound(rmid); + return RmgrTable[rmid]; +} /* * Exported to support xlog switching from checkpointer -- 2.17.1
