Hi, This feature adds an option to skip changes of all tables in specified schema while creating publication. This feature is helpful for use cases where the user wants to subscribe to all the changes except for the changes present in a few schemas. Ex: CREATE PUBLICATION pub1 FOR ALL TABLES SKIP ALL TABLES IN SCHEMA s1,s2; OR ALTER PUBLICATION pub1 ADD SKIP ALL TABLES IN SCHEMA s1,s2;
A new column pnskip is added to table "pg_publication_namespace", to maintain the schemas that the user wants to skip publishing through the publication. Modified the output plugin (pgoutput) to skip publishing the changes if the relation is part of skip schema publication. As a continuation to this, I will work on implementing skipping tables from all tables in schema and skipping tables from all tables publication. Attached patch has the implementation for this. This feature is for the pg16 version. Thoughts? Regards, Vignesh
From 153e033a78ace66bfbe1cd37db2f3de506740bcb Mon Sep 17 00:00:00 2001 From: Vigneshwaran C <vignes...@gmail.com> Date: Fri, 18 Mar 2022 10:41:35 +0530 Subject: [PATCH v1] Skip publishing the tables of schema. A new option "SKIP ALL TABLES IN SCHEMA" in Create/Alter Publication allows one or more skip schemas to be specified, publisher will skip sending the data of the tables present in the skip schema to the subscriber. The new syntax allows specifying schemas. For example: CREATE PUBLICATION pub1 FOR ALL TABLES SKIP ALL TABLES IN SCHEMA s1,s2; OR ALTER PUBLICATION pub1 ADD SKIP ALL TABLES IN SCHEMA s1,s2; A new column pnskip is added to table "pg_publication_namespace", to maintain the schemas that the user wants to skip publishing through the publication. Modified the output plugin (pgoutput) to skip publishing the changes if the relation is part of skip schema publication. Updates pg_dump to identify and dump skip schema publications. Updates the \d family of commands to display skip schema publications and \dRp+ variant will now display associated skip schemas if any. --- doc/src/sgml/catalogs.sgml | 9 ++ doc/src/sgml/logical-replication.sgml | 7 +- doc/src/sgml/ref/alter_publication.sgml | 28 +++- doc/src/sgml/ref/create_publication.sgml | 28 +++- doc/src/sgml/ref/psql-ref.sgml | 5 +- src/backend/catalog/pg_publication.c | 66 +++++++--- src/backend/commands/publicationcmds.c | 123 +++++++++++------- src/backend/commands/tablecmds.c | 2 +- src/backend/nodes/copyfuncs.c | 14 ++ src/backend/nodes/equalfuncs.c | 14 ++ src/backend/parser/gram.y | 99 +++++++++++++- src/backend/replication/pgoutput/pgoutput.c | 24 +--- src/backend/utils/cache/relcache.c | 22 +++- src/bin/pg_dump/pg_dump.c | 33 ++++- src/bin/pg_dump/pg_dump.h | 1 + src/bin/pg_dump/pg_dump_sort.c | 7 + src/bin/pg_dump/t/002_pg_dump.pl | 30 +++++ src/bin/psql/describe.c | 17 +++ src/bin/psql/tab-complete.c | 25 ++-- src/include/catalog/pg_publication.h | 20 ++- .../catalog/pg_publication_namespace.h | 1 + src/include/commands/publicationcmds.h | 3 +- src/include/nodes/nodes.h | 1 + src/include/nodes/parsenodes.h | 1 + src/test/regress/expected/publication.out | 84 +++++++++++- src/test/regress/sql/publication.sql | 41 +++++- .../t/030_rep_changes_skip_schema.pl | 96 ++++++++++++++ src/tools/pgindent/typedefs.list | 1 + 28 files changed, 678 insertions(+), 124 deletions(-) create mode 100644 src/test/subscription/t/030_rep_changes_skip_schema.pl diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 2a8cd02664..18e3cf82aa 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -6281,6 +6281,15 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l Reference to schema </para></entry> </row> + + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>pnskip</structfield> <type>bool</type> + </para> + <para> + True if the schema is skip schema + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/doc/src/sgml/logical-replication.sgml b/doc/src/sgml/logical-replication.sgml index 555fbd749c..e2a4b89226 100644 --- a/doc/src/sgml/logical-replication.sgml +++ b/doc/src/sgml/logical-replication.sgml @@ -599,9 +599,10 @@ CONTEXT: processing remote data for replication origin "pg_16395" during "INSER <para> To add tables to a publication, the user must have ownership rights on the - table. To add all tables in schema to a publication, the user must be a - superuser. To create a publication that publishes all tables or all tables in - schema automatically, the user must be a superuser. + table. To add all tables in schema or skip all tables in schema to a + publication, the user must be a superuser. To create a publication that + publishes all tables or all tables in schema automatically, the user must be + a superuser. </para> <para> diff --git a/doc/src/sgml/ref/alter_publication.sgml b/doc/src/sgml/ref/alter_publication.sgml index 32b75f6c78..8466e94ab0 100644 --- a/doc/src/sgml/ref/alter_publication.sgml +++ b/doc/src/sgml/ref/alter_publication.sgml @@ -31,7 +31,7 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RENAME TO <r <phrase>where <replaceable class="parameter">publication_object</replaceable> is one of:</phrase> TABLE [ ONLY ] <replaceable class="parameter">table_name</replaceable> [ * ] [ WHERE ( <replaceable class="parameter">expression</replaceable> ) ] [, ... ] - ALL TABLES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ] + [SKIP] ALL TABLES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA } [, ... ] </synopsis> </refsynopsisdiv> @@ -71,12 +71,12 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RENAME TO <r <para> You must own the publication to use <command>ALTER PUBLICATION</command>. Adding a table to a publication additionally requires owning that table. - The <literal>ADD ALL TABLES IN SCHEMA</literal> and - <literal>SET ALL TABLES IN SCHEMA</literal> to a publication requires the - invoking user to be a superuser. To alter the owner, you must also be a + The <literal>ADD [SKIP] ALL TABLES IN SCHEMA</literal> and + <literal>SET [SKIP] ALL TABLES IN SCHEMA</literal> to a publication requires + the invoking user to be a superuser. To alter the owner, you must also be a direct or indirect member of the new owning role. The new owner must have <literal>CREATE</literal> privilege on the database. Also, the new owner - of a <literal>FOR ALL TABLES</literal> or <literal>FOR ALL TABLES IN + of a <literal>FOR ALL TABLES</literal> or <literal>FOR [SKIP] ALL TABLES IN SCHEMA</literal> publication must be a superuser. However, a superuser can change the ownership of a publication regardless of these restrictions. </para> @@ -88,6 +88,14 @@ ALTER PUBLICATION <replaceable class="parameter">name</replaceable> RENAME TO <r adding/setting a table to a publication that already has a table's schema as part of the specified schema is not supported. </para> + + <para> + The <literal>ADD SKIP ALL TABLES IN SCHEMA</literal> and + <literal>SET SKIP ALL TABLES IN SCHEMA</literal> can be specified only for + <literal>FOR ALL TABLES</literal> publication. It is not supported for + <literal>FOR ALL TABLES IN SCHEMA </literal> publication and + <literal>FOR TABLE</literal> publication. + </para> </refsect1> <refsect1> @@ -195,6 +203,16 @@ ALTER PUBLICATION sales_publication ADD ALL TABLES IN SCHEMA marketing, sales; ALTER PUBLICATION production_publication ADD TABLE users, departments, ALL TABLES IN SCHEMA production; </programlisting> </para> + + <para> + Add skip schemas <structname>sales_june</structname> and + <structname>sales_july</structname> to the publication + <structname>mypublication</structname>: +<programlisting> +ALTER PUBLICATION mypublication ADD SKIP ALL TABLES IN SCHEMA sales_june, sales_july; +</programlisting> + </para> + </refsect1> <refsect1> diff --git a/doc/src/sgml/ref/create_publication.sgml b/doc/src/sgml/ref/create_publication.sgml index 4979b9b646..03d8e82eec 100644 --- a/doc/src/sgml/ref/create_publication.sgml +++ b/doc/src/sgml/ref/create_publication.sgml @@ -22,7 +22,7 @@ PostgreSQL documentation <refsynopsisdiv> <synopsis> CREATE PUBLICATION <replaceable class="parameter">name</replaceable> - [ FOR ALL TABLES + [ FOR ALL TABLES [SKIP ALL TABLES IN SCHEMA { <replaceable class="parameter">schema_name</replaceable> | CURRENT_SCHEMA }] | FOR <replaceable class="parameter">publication_object</replaceable> [, ... ] ] [ WITH ( <replaceable class="parameter">publication_parameter</replaceable> [= <replaceable class="parameter">value</replaceable>] [, ... ] ) ] @@ -117,6 +117,23 @@ CREATE PUBLICATION <replaceable class="parameter">name</replaceable> </listitem> </varlistentry> + <varlistentry> + <term><literal>SKIP ALL TABLES IN SCHEMA</literal></term> + <listitem> + <para> + Marks the publication as one that skips replicating changes for all + tables in the specified list of schemas. + </para> + + <para> + <literal>SKIP ALL TABLES IN SCHEMA</literal> can be specified only for + <literal>FOR ALL TABLES</literal> publication. It is not supported for + <literal>FOR ALL TABLES IN SCHEMA </literal> publication and + <literal>FOR TABLE</literal> publication. + </para> + </listitem> + </varlistentry> + <varlistentry> <term><literal>FOR ALL TABLES IN SCHEMA</literal></term> <listitem> @@ -327,6 +344,15 @@ CREATE PUBLICATION production_publication FOR TABLE users, departments, ALL TABL <structname>sales</structname>: <programlisting> CREATE PUBLICATION sales_publication FOR ALL TABLES IN SCHEMA marketing, sales; +</programlisting> + </para> + + <para> + Create a publication that publishes all changes in all the tables except for + the changes of all the tables present in the schema + <structname>marketing</structname> and <structname>sales</structname>: +<programlisting> +CREATE PUBLICATION mypublication FOR ALL TABLE SKIP ALL TABLES IN SCHEMA marketing, sales; </programlisting></para> </refsect1> diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml index caabb06c53..4ba4140933 100644 --- a/doc/src/sgml/ref/psql-ref.sgml +++ b/doc/src/sgml/ref/psql-ref.sgml @@ -1856,8 +1856,9 @@ testdb=> If <replaceable class="parameter">pattern</replaceable> is specified, only those publications whose names match the pattern are listed. - If <literal>+</literal> is appended to the command name, the tables and - schemas associated with each publication are shown as well. + If <literal>+</literal> is appended to the command name, the tables, + schemas and the skip schema associated with each publication are shown + as well. </para> </listitem> </varlistentry> diff --git a/src/backend/catalog/pg_publication.c b/src/backend/catalog/pg_publication.c index 789b895db8..711dda864f 100644 --- a/src/backend/catalog/pg_publication.c +++ b/src/backend/catalog/pg_publication.c @@ -287,7 +287,8 @@ GetPubPartitionOptionRelations(List *result, PublicationPartOpt pub_partopt, * ancestor is at the end of the list. */ Oid -GetTopMostAncestorInPublication(Oid puboid, List *ancestors, int *ancestor_level) +GetTopMostAncestorInPublication(Oid puboid, List *ancestors, + int *ancestor_level, bool puballtables) { ListCell *lc; Oid topmost_relid = InvalidOid; @@ -301,6 +302,7 @@ GetTopMostAncestorInPublication(Oid puboid, List *ancestors, int *ancestor_level Oid ancestor = lfirst_oid(lc); List *apubids = GetRelationPublications(ancestor); List *aschemaPubids = NIL; + List *askipschemaPubids = NIL; level++; @@ -313,8 +315,10 @@ GetTopMostAncestorInPublication(Oid puboid, List *ancestors, int *ancestor_level } else { - aschemaPubids = GetSchemaPublications(get_rel_namespace(ancestor)); - if (list_member_oid(aschemaPubids, puboid)) + aschemaPubids = GetSchemaPublications(get_rel_namespace(ancestor), false); + askipschemaPubids = GetSchemaPublications(get_rel_namespace(ancestor), true); + if (list_member_oid(aschemaPubids, puboid) || + (puballtables && !list_member_oid(askipschemaPubids, puboid))) { topmost_relid = ancestor; @@ -436,13 +440,14 @@ publication_add_relation(Oid pubid, PublicationRelInfo *pri, * Insert new publication / schema mapping. */ ObjectAddress -publication_add_schema(Oid pubid, Oid schemaid, bool if_not_exists) +publication_add_schema(Oid pubid, PublicationSchInfo *pubsch, bool if_not_exists) { Relation rel; HeapTuple tup; Datum values[Natts_pg_publication_namespace]; bool nulls[Natts_pg_publication_namespace]; Oid psschid; + Oid schemaid = pubsch->oid; Publication *pub = GetPublication(pubid); List *schemaRels = NIL; ObjectAddress myself, @@ -483,6 +488,8 @@ publication_add_schema(Oid pubid, Oid schemaid, bool if_not_exists) ObjectIdGetDatum(pubid); values[Anum_pg_publication_namespace_pnnspid - 1] = ObjectIdGetDatum(schemaid); + values[Anum_pg_publication_namespace_pnskip - 1] = + BoolGetDatum(pubsch->skip); tup = heap_form_tuple(RelationGetDescr(rel), values, nulls); @@ -630,13 +637,23 @@ GetAllTablesPublications(void) * root partitioned tables. */ List * -GetAllTablesPublicationRelations(bool pubviaroot) +GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot) { Relation classRel; ScanKeyData key[1]; TableScanDesc scan; HeapTuple tuple; List *result = NIL; + List *skipschemaidlist = NIL; + List *pubschemalist = GetPublicationSchemas(pubid); + ListCell *cell; + + foreach(cell, pubschemalist) + { + PublicationSchInfo *pubsch = (PublicationSchInfo *) lfirst(cell); + + skipschemaidlist = lappend_oid(result, pubsch->oid); + } classRel = table_open(RelationRelationId, AccessShareLock); @@ -651,9 +668,11 @@ GetAllTablesPublicationRelations(bool pubviaroot) { Form_pg_class relForm = (Form_pg_class) GETSTRUCT(tuple); Oid relid = relForm->oid; + Oid schid = get_rel_namespace(relid); if (is_publishable_class(relid, relForm) && - !(relForm->relispartition && pubviaroot)) + !(relForm->relispartition && pubviaroot) && + !list_member_oid(skipschemaidlist, schid)) result = lappend_oid(result, relid); } @@ -672,9 +691,11 @@ GetAllTablesPublicationRelations(bool pubviaroot) { Form_pg_class relForm = (Form_pg_class) GETSTRUCT(tuple); Oid relid = relForm->oid; + Oid schid = get_rel_namespace(relid); if (is_publishable_class(relid, relForm) && - !relForm->relispartition) + !relForm->relispartition && + !list_member_oid(skipschemaidlist, schid)) result = lappend_oid(result, relid); } @@ -713,10 +734,14 @@ GetPublicationSchemas(Oid pubid) while (HeapTupleIsValid(tup = systable_getnext(scan))) { Form_pg_publication_namespace pubsch; + PublicationSchInfo *schinfo = makeNode(PublicationSchInfo); + pubsch = (Form_pg_publication_namespace) GETSTRUCT(tup); + schinfo->oid = pubsch->pnnspid; + schinfo->skip = pubsch->pnskip; - result = lappend_oid(result, pubsch->pnnspid); + result = lappend(result, schinfo); } systable_endscan(scan); @@ -729,7 +754,7 @@ GetPublicationSchemas(Oid pubid) * Gets the list of publication oids associated with a specified schema. */ List * -GetSchemaPublications(Oid schemaid) +GetSchemaPublications(Oid schemaid, bool skippub) { List *result = NIL; CatCList *pubschlist; @@ -743,7 +768,8 @@ GetSchemaPublications(Oid schemaid) HeapTuple tup = &pubschlist->members[i]->tuple; Oid pubid = ((Form_pg_publication_namespace) GETSTRUCT(tup))->pnpubid; - result = lappend_oid(result, pubid); + if (skippub == ((Form_pg_publication_namespace) GETSTRUCT(tup))->pnskip) + result = lappend_oid(result, pubid); } ReleaseSysCacheList(pubschlist); @@ -812,7 +838,8 @@ GetSchemaPublicationRelations(Oid schemaid, PublicationPartOpt pub_partopt) * publication. */ List * -GetAllSchemaPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt) +GetAllSchemaPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt, + bool bskip) { List *result = NIL; List *pubschemalist = GetPublicationSchemas(pubid); @@ -820,11 +847,16 @@ GetAllSchemaPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt) foreach(cell, pubschemalist) { - Oid schemaid = lfirst_oid(cell); + PublicationSchInfo *pubsch = (PublicationSchInfo *) lfirst(cell); List *schemaRels = NIL; - schemaRels = GetSchemaPublicationRelations(schemaid, pub_partopt); - result = list_concat(result, schemaRels); + + /* Skip the skip publication schemas if bskip is true */ + if (bskip && !pubsch->skip) + { + schemaRels = GetSchemaPublicationRelations(pubsch->oid, pub_partopt); + result = list_concat(result, schemaRels); + } } return result; @@ -958,7 +990,8 @@ pg_get_publication_tables(PG_FUNCTION_ARGS) */ if (publication->alltables) { - tables = GetAllTablesPublicationRelations(publication->pubviaroot); + tables = GetAllTablesPublicationRelations(publication->oid, + publication->pubviaroot); } else { @@ -972,7 +1005,8 @@ pg_get_publication_tables(PG_FUNCTION_ARGS) schemarelids = GetAllSchemaPublicationRelations(publication->oid, publication->pubviaroot ? PUBLICATION_PART_ROOT : - PUBLICATION_PART_LEAF); + PUBLICATION_PART_LEAF, + true); tables = list_concat_unique_oid(relids, schemarelids); /* diff --git a/src/backend/commands/publicationcmds.c b/src/backend/commands/publicationcmds.c index 1aad2e769c..8fd63c01a9 100644 --- a/src/backend/commands/publicationcmds.c +++ b/src/backend/commands/publicationcmds.c @@ -177,8 +177,8 @@ ObjectsInPublicationToOids(List *pubobjspec_list, ParseState *pstate, foreach(cell, pubobjspec_list) { - Oid schemaid; List *search_path; + PublicationSchInfo *pubsch = makeNode(PublicationSchInfo); pubobj = (PublicationObjSpec *) lfirst(cell); @@ -188,10 +188,11 @@ ObjectsInPublicationToOids(List *pubobjspec_list, ParseState *pstate, *rels = lappend(*rels, pubobj->pubtable); break; case PUBLICATIONOBJ_TABLES_IN_SCHEMA: - schemaid = get_namespace_oid(pubobj->name, false); + pubsch->oid = get_namespace_oid(pubobj->name, false); + pubsch->skip = pubobj->skip; /* Filter out duplicates if user specifies "sch1, sch1" */ - *schemas = list_append_unique_oid(*schemas, schemaid); + *schemas = list_append_unique(*schemas, pubsch); break; case PUBLICATIONOBJ_TABLES_IN_CUR_SCHEMA: search_path = fetch_search_path(false); @@ -200,11 +201,11 @@ ObjectsInPublicationToOids(List *pubobjspec_list, ParseState *pstate, errcode(ERRCODE_UNDEFINED_SCHEMA), errmsg("no schema has been selected for CURRENT_SCHEMA")); - schemaid = linitial_oid(search_path); + pubsch->oid = linitial_oid(search_path); list_free(search_path); /* Filter out duplicates if user specifies "sch1, sch1" */ - *schemas = list_append_unique_oid(*schemas, schemaid); + *schemas = list_append_unique(*schemas, pubsch); break; default: /* shouldn't happen */ @@ -230,24 +231,29 @@ CheckObjSchemaNotAlreadyInPublication(List *rels, List *schemaidlist, Relation rel = pub_rel->relation; Oid relSchemaId = RelationGetNamespace(rel); - if (list_member_oid(schemaidlist, relSchemaId)) + foreach(lc, schemaidlist) { - if (checkobjtype == PUBLICATIONOBJ_TABLES_IN_SCHEMA) - ereport(ERROR, - errcode(ERRCODE_INVALID_PARAMETER_VALUE), - errmsg("cannot add schema \"%s\" to publication", - get_namespace_name(relSchemaId)), - errdetail("Table \"%s\" in schema \"%s\" is already part of the publication, adding the same schema is not supported.", - RelationGetRelationName(rel), - get_namespace_name(relSchemaId))); - else if (checkobjtype == PUBLICATIONOBJ_TABLE) - ereport(ERROR, - errcode(ERRCODE_INVALID_PARAMETER_VALUE), - errmsg("cannot add relation \"%s.%s\" to publication", - get_namespace_name(relSchemaId), - RelationGetRelationName(rel)), - errdetail("Table's schema \"%s\" is already part of the publication or part of the specified schema list.", - get_namespace_name(relSchemaId))); + PublicationSchInfo *pub_sch = (PublicationSchInfo *) lfirst(lc); + + if (pub_sch->oid == relSchemaId) + { + if (checkobjtype == PUBLICATIONOBJ_TABLES_IN_SCHEMA) + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("cannot add schema \"%s\" to publication", + get_namespace_name(relSchemaId)), + errdetail("Table \"%s\" in schema \"%s\" is already part of the publication, adding the same schema is not supported.", + RelationGetRelationName(rel), + get_namespace_name(relSchemaId))); + else if (checkobjtype == PUBLICATIONOBJ_TABLE) + ereport(ERROR, + errcode(ERRCODE_INVALID_PARAMETER_VALUE), + errmsg("cannot add relation \"%s.%s\" to publication", + get_namespace_name(relSchemaId), + RelationGetRelationName(rel)), + errdetail("Table's schema \"%s\" is already part of the publication or part of the specified schema list.", + get_namespace_name(relSchemaId))); + } } } } @@ -297,7 +303,7 @@ contain_invalid_rfcolumn_walker(Node *node, rf_context *context) */ bool contain_invalid_rfcolumn(Oid pubid, Relation relation, List *ancestors, - bool pubviaroot) + bool pubviaroot, bool puballtables) { HeapTuple rftuple; Oid relid = RelationGetRelid(relation); @@ -324,7 +330,8 @@ contain_invalid_rfcolumn(Oid pubid, Relation relation, List *ancestors, if (pubviaroot && relation->rd_rel->relispartition) { publish_as_relid - = GetTopMostAncestorInPublication(pubid, ancestors, NULL); + = GetTopMostAncestorInPublication(pubid, ancestors, NULL, + puballtables); if (!OidIsValid(publish_as_relid)) publish_as_relid = relid; @@ -697,18 +704,20 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt) /* Make the changes visible. */ CommandCounterIncrement(); + ObjectsInPublicationToOids(stmt->pubobjects, pstate, &relations, + &schemaidlist); + /* Associate objects with the publication. */ if (stmt->for_all_tables) { + Assert(!relations); + /* Invalidate relcache so that publication info is rebuilt. */ CacheInvalidateRelcacheAll(); } else { - ObjectsInPublicationToOids(stmt->pubobjects, pstate, &relations, - &schemaidlist); - - /* FOR ALL TABLES IN SCHEMA requires superuser */ + /* FOR [SKIP] ALL TABLES IN SCHEMA requires superuser */ if (list_length(schemaidlist) > 0 && !superuser()) ereport(ERROR, errcode(ERRCODE_INSUFFICIENT_PRIVILEGE), @@ -728,16 +737,16 @@ CreatePublication(ParseState *pstate, CreatePublicationStmt *stmt) PublicationAddTables(puboid, rels, true, NULL); CloseTableList(rels); } + } - if (list_length(schemaidlist) > 0) - { - /* - * Schema lock is held until the publication is created to prevent - * concurrent schema deletion. - */ - LockSchemaList(schemaidlist); - PublicationAddSchemas(puboid, schemaidlist, true, NULL); - } + if (list_length(schemaidlist) > 0) + { + /* + * Schema lock is held until the publication is created to prevent + * concurrent schema deletion. + */ + LockSchemaList(schemaidlist); + PublicationAddSchemas(puboid, schemaidlist, true, NULL); } table_close(rel, RowExclusiveLock); @@ -906,7 +915,8 @@ AlterPublicationOptions(ParseState *pstate, AlterPublicationStmt *stmt, } schemarelids = GetAllSchemaPublicationRelations(pubform->oid, - PUBLICATION_PART_ALL); + PUBLICATION_PART_ALL, + false); relids = list_concat_unique_oid(relids, schemarelids); InvalidatePublicationRels(relids); @@ -970,7 +980,7 @@ AlterPublicationTables(AlterPublicationStmt *stmt, HeapTuple tup, * Check if the relation is member of the existing schema in the * publication or member of the schema list specified. */ - schemas = list_concat_copy(schemaidlist, GetPublicationSchemas(pubid)); + schemas = list_concat(schemaidlist, GetPublicationSchemas(pubid)); CheckObjSchemaNotAlreadyInPublication(rels, schemas, PUBLICATIONOBJ_TABLE); @@ -1124,7 +1134,7 @@ AlterPublicationSchemas(AlterPublicationStmt *stmt, List *delschemas = NIL; /* Identify which schemas should be dropped */ - delschemas = list_difference_oid(oldschemaids, schemaidlist); + delschemas = list_difference(oldschemaids, schemaidlist); /* * Schema lock is held until the publication is altered to prevent @@ -1152,6 +1162,20 @@ CheckAlterPublication(AlterPublicationStmt *stmt, HeapTuple tup, List *tables, List *schemaidlist) { Form_pg_publication pubform = (Form_pg_publication) GETSTRUCT(tup); + ListCell *lc; + + bool nonskipschema = false; + bool skipschema = false; + + foreach(lc, schemaidlist) + { + PublicationSchInfo *pub_sch = (PublicationSchInfo *) lfirst(lc); + + if (!pub_sch->skip) + nonskipschema = true; + else + skipschema = true; + } if ((stmt->action == AP_AddObjects || stmt->action == AP_SetObjects) && schemaidlist && !superuser()) @@ -1163,13 +1187,20 @@ CheckAlterPublication(AlterPublicationStmt *stmt, HeapTuple tup, * Check that user is allowed to manipulate the publication tables in * schema */ - if (schemaidlist && pubform->puballtables) + if (nonskipschema && pubform->puballtables) ereport(ERROR, (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), errmsg("publication \"%s\" is defined as FOR ALL TABLES", NameStr(pubform->pubname)), errdetail("Tables from schema cannot be added to, dropped from, or set on FOR ALL TABLES publications."))); + if (skipschema && !pubform->puballtables) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("publication \"%s\" is not defined as FOR ALL TABLES", + NameStr(pubform->pubname)), + errdetail("Skip tables from schema cannot be added to, dropped from, or set on NON ALL TABLES publications."))); + /* Check that user is allowed to manipulate the publication tables. */ if (tables && pubform->puballtables) ereport(ERROR, @@ -1543,7 +1574,8 @@ LockSchemaList(List *schemalist) foreach(lc, schemalist) { - Oid schemaid = lfirst_oid(lc); + PublicationSchInfo *pubsch = (PublicationSchInfo *) lfirst(lc); + Oid schemaid = pubsch->oid; /* Allow query cancel in case this takes a long time */ CHECK_FOR_INTERRUPTS(); @@ -1648,10 +1680,10 @@ PublicationAddSchemas(Oid pubid, List *schemas, bool if_not_exists, foreach(lc, schemas) { - Oid schemaid = lfirst_oid(lc); ObjectAddress obj; + PublicationSchInfo *pubsch = (PublicationSchInfo *) lfirst(lc); - obj = publication_add_schema(pubid, schemaid, if_not_exists); + obj = publication_add_schema(pubid, pubsch, if_not_exists); if (stmt) { EventTriggerCollectSimpleCommand(obj, InvalidObjectAddress, @@ -1675,7 +1707,8 @@ PublicationDropSchemas(Oid pubid, List *schemas, bool missing_ok) foreach(lc, schemas) { - Oid schemaid = lfirst_oid(lc); + PublicationSchInfo *pubsch = (PublicationSchInfo *) lfirst(lc); + Oid schemaid = pubsch->oid; psid = GetSysCacheOid2(PUBLICATIONNAMESPACEMAP, Anum_pg_publication_namespace_oid, diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 80faae985e..e4ae136ac1 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -16385,7 +16385,7 @@ AlterTableNamespace(AlterObjectSchemaStmt *stmt, Oid *oldschema) if (stmt->objectType == OBJECT_TABLE) { ListCell *lc; - List *schemaPubids = GetSchemaPublications(nspOid); + List *schemaPubids = GetSchemaPublications(nspOid, false); List *relPubids = GetRelationPublications(RelationGetRelid(rel)); foreach(lc, relPubids) diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c index d4f8455a2b..81ea805826 100644 --- a/src/backend/nodes/copyfuncs.c +++ b/src/backend/nodes/copyfuncs.c @@ -4854,6 +4854,17 @@ _copyPublicationTable(const PublicationTable *from) return newnode; } +static PublicationSchInfo * +_copyPublicationSchInfo(const PublicationSchInfo *from) +{ + PublicationSchInfo *newnode = makeNode(PublicationSchInfo); + + COPY_SCALAR_FIELD(oid); + COPY_SCALAR_FIELD(skip); + + return newnode; +} + static CreatePublicationStmt * _copyCreatePublicationStmt(const CreatePublicationStmt *from) { @@ -5940,6 +5951,9 @@ copyObjectImpl(const void *from) case T_PublicationObjSpec: retval = _copyPublicationObject(from); break; + case T_PublicationSchInfo: + retval = _copyPublicationSchInfo(from); + break; case T_PublicationTable: retval = _copyPublicationTable(from); break; diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c index f1002afe7a..b2b911a94f 100644 --- a/src/backend/nodes/equalfuncs.c +++ b/src/backend/nodes/equalfuncs.c @@ -33,6 +33,7 @@ #include "nodes/extensible.h" #include "nodes/pathnodes.h" #include "utils/datum.h" +#include "catalog/pg_publication.h" /* @@ -2326,6 +2327,16 @@ _equalPublicationTable(const PublicationTable *a, const PublicationTable *b) return true; } +static bool +_equalPublicationSchema(const PublicationSchInfo *a, + const PublicationSchInfo *b) +{ + COMPARE_SCALAR_FIELD(oid); + COMPARE_SCALAR_FIELD(skip); + + return true; +} + static bool _equalCreatePublicationStmt(const CreatePublicationStmt *a, const CreatePublicationStmt *b) @@ -3935,6 +3946,9 @@ equal(const void *a, const void *b) case T_PublicationObjSpec: retval = _equalPublicationObject(a, b); break; + case T_PublicationSchInfo: + retval = _equalPublicationSchema(a, b); + break; case T_PublicationTable: retval = _equalPublicationTable(a, b); break; diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0036c2f9e2..239013af91 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -219,6 +219,10 @@ static void processCASbits(int cas_bits, int location, const char *constrType, bool *no_inherit, core_yyscan_t yyscanner); static void preprocess_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner); +static void preprocess_alltables_pubobj_list(List *pubobjspec_list, + core_yyscan_t yyscanner); +static void check_skip_in_pubobj_list(List *pubobjspec_list, + core_yyscan_t yyscanner); static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); %} @@ -446,7 +450,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query); transform_element_list transform_type_list TriggerTransitions TriggerReferencing vacuum_relation_list opt_vacuum_relation_list - drop_option_list pub_obj_list + drop_option_list pub_obj_list skip_pub_obj_list %type <node> opt_routine_body %type <groupclause> group_clause @@ -9718,12 +9722,15 @@ CreatePublicationStmt: n->options = $4; $$ = (Node *)n; } - | CREATE PUBLICATION name FOR ALL TABLES opt_definition + | CREATE PUBLICATION name FOR ALL TABLES skip_pub_obj_list opt_definition { CreatePublicationStmt *n = makeNode(CreatePublicationStmt); n->pubname = $3; - n->options = $7; + n->options = $8; n->for_all_tables = true; + n->pubobjects = (List *)$7; + preprocess_pubobj_list(n->pubobjects, yyscanner); + preprocess_alltables_pubobj_list(n->pubobjects, yyscanner); $$ = (Node *)n; } | CREATE PUBLICATION name FOR pub_obj_list opt_definition @@ -9733,6 +9740,7 @@ CreatePublicationStmt: n->options = $6; n->pubobjects = (List *)$5; preprocess_pubobj_list(n->pubobjects, yyscanner); + check_skip_in_pubobj_list(n->pubobjects, yyscanner); $$ = (Node *)n; } ; @@ -9764,14 +9772,31 @@ PublicationObjSpec: $$ = makeNode(PublicationObjSpec); $$->pubobjtype = PUBLICATIONOBJ_TABLES_IN_SCHEMA; $$->name = $5; + $$->skip = false; $$->location = @5; } | ALL TABLES IN_P SCHEMA CURRENT_SCHEMA { $$ = makeNode(PublicationObjSpec); $$->pubobjtype = PUBLICATIONOBJ_TABLES_IN_CUR_SCHEMA; + $$->skip = false; $$->location = @5; } + | SKIP ALL TABLES IN_P SCHEMA ColId + { + $$ = makeNode(PublicationObjSpec); + $$->pubobjtype = PUBLICATIONOBJ_TABLES_IN_SCHEMA; + $$->name = $6; + $$->skip = true; + $$->location = @6; + } + | SKIP ALL TABLES IN_P SCHEMA CURRENT_SCHEMA + { + $$ = makeNode(PublicationObjSpec); + $$->pubobjtype = PUBLICATIONOBJ_TABLES_IN_CUR_SCHEMA; + $$->skip = true; + $$->location = @6; + } | ColId OptWhereClause { $$ = makeNode(PublicationObjSpec); @@ -9826,6 +9851,12 @@ pub_obj_list: PublicationObjSpec { $$ = lappend($1, $3); } ; + skip_pub_obj_list: pub_obj_list + { $$ = $1; } + | /*EMPTY*/ + { $$ = NULL; } + ; + /***************************************************************************** * * ALTER PUBLICATION name SET ( options ) @@ -17448,6 +17479,7 @@ preprocess_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner) ListCell *cell; PublicationObjSpec *pubobj; PublicationObjSpecType prevobjtype = PUBLICATIONOBJ_CONTINUATION; + bool prevskipobj = false; if (!pubobjspec_list) return; @@ -17465,7 +17497,10 @@ preprocess_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner) pubobj = (PublicationObjSpec *) lfirst(cell); if (pubobj->pubobjtype == PUBLICATIONOBJ_CONTINUATION) + { pubobj->pubobjtype = prevobjtype; + pubobj->skip = prevskipobj; + } if (pubobj->pubobjtype == PUBLICATIONOBJ_TABLE) { @@ -17513,6 +17548,64 @@ preprocess_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner) } prevobjtype = pubobj->pubobjtype; + prevskipobj = pubobj->skip; + } +} + +/* + * Process pubobjspec_list to check if any other option other that + * "SKIP ALL TABLES IN SCHEMA" is specified with "ALL TABLES" and throw an + * error. + */ +static void +preprocess_alltables_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner) +{ + ListCell *cell; + PublicationObjSpec *pubobj; + + if (!pubobjspec_list) + return; + + foreach(cell, pubobjspec_list) + { + pubobj = (PublicationObjSpec *) lfirst(cell); + + /* Only SKIP ALL TABLES IN SCHEMA option supported with ALL TABLES */ + if (pubobj->pubobjtype != PUBLICATIONOBJ_TABLES_IN_SCHEMA || + !pubobj->skip) + { + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("only SKIP ALL TABLES IN SCHEMA can be specified with ALL TABLES option"), + parser_errposition(pubobj->location)); + } + } +} + +/* + * Process pubobjspec_list to check if "SKIP ALL TABLES IN SCHEMA" is specified + * with "ALL TABLES" and throw an error. + */ +static void +check_skip_in_pubobj_list(List *pubobjspec_list, core_yyscan_t yyscanner) +{ + ListCell *cell; + PublicationObjSpec *pubobj; + + if (!pubobjspec_list) + return; + + foreach(cell, pubobjspec_list) + { + pubobj = (PublicationObjSpec *) lfirst(cell); + + /* Only SKIP ALL TABLES IN SCHEMA option supported with ALL TABLES */ + if (pubobj->pubobjtype == PUBLICATIONOBJ_TABLES_IN_SCHEMA && + pubobj->skip) + ereport(ERROR, + errcode(ERRCODE_SYNTAX_ERROR), + errmsg("SKIP ALL TABLES IN SCHEMA can be specified only with ALL TABLES option"), + parser_errposition(pubobj->location)); } } diff --git a/src/backend/replication/pgoutput/pgoutput.c b/src/backend/replication/pgoutput/pgoutput.c index 5fddab3a3d..c1564394a8 100644 --- a/src/backend/replication/pgoutput/pgoutput.c +++ b/src/backend/replication/pgoutput/pgoutput.c @@ -1745,7 +1745,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation) * the cache entry using a historic snapshot and all the later changes * are absorbed while decoding WAL. */ - List *schemaPubids = GetSchemaPublications(schemaId); + List *schemaPubids = GetSchemaPublications(schemaId, false); + List *skipSchemaPubids = GetSchemaPublications(schemaId, true); ListCell *lc; Oid publish_as_relid = relid; int publish_ancestor_level = 0; @@ -1824,22 +1825,6 @@ get_rel_sync_entry(PGOutputData *data, Relation relation) Oid pub_relid = relid; int ancestor_level = 0; - /* - * If this is a FOR ALL TABLES publication, pick the partition root - * and set the ancestor level accordingly. - */ - if (pub->alltables) - { - publish = true; - if (pub->pubviaroot && am_partition) - { - List *ancestors = get_partition_ancestors(relid); - - pub_relid = llast_oid(ancestors); - ancestor_level = list_length(ancestors); - } - } - if (!publish) { bool ancestor_published = false; @@ -1858,7 +1843,8 @@ get_rel_sync_entry(PGOutputData *data, Relation relation) ancestor = GetTopMostAncestorInPublication(pub->oid, ancestors, - &level); + &level, + pub->alltables); if (ancestor != InvalidOid) { @@ -1873,6 +1859,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation) if (list_member_oid(pubids, pub->oid) || list_member_oid(schemaPubids, pub->oid) || + (pub->alltables && !list_member_oid(skipSchemaPubids, pub->oid)) || ancestor_published) publish = true; } @@ -1942,6 +1929,7 @@ get_rel_sync_entry(PGOutputData *data, Relation relation) list_free(pubids); list_free(schemaPubids); + list_free(skipSchemaPubids); list_free(rel_publications); entry->replicate_valid = true; diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c index fccffce572..5771906589 100644 --- a/src/backend/utils/cache/relcache.c +++ b/src/backend/utils/cache/relcache.c @@ -5538,6 +5538,8 @@ void RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc) { List *puboids; + List *alltablespuboids; + List *skipschemapuboids; ListCell *lc; MemoryContext oldcxt; Oid schemaid; @@ -5569,7 +5571,8 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc) /* Fetch the publication membership info. */ puboids = GetRelationPublications(relid); schemaid = RelationGetNamespace(relation); - puboids = list_concat_unique_oid(puboids, GetSchemaPublications(schemaid)); + puboids = list_concat_unique_oid(puboids, GetSchemaPublications(schemaid, false)); + skipschemapuboids = GetSchemaPublications(schemaid, true); if (relation->rd_rel->relispartition) { @@ -5584,11 +5587,21 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc) GetRelationPublications(ancestor)); schemaid = get_rel_namespace(ancestor); puboids = list_concat_unique_oid(puboids, - GetSchemaPublications(schemaid)); + GetSchemaPublications(schemaid, false)); + skipschemapuboids = list_concat_unique_oid(skipschemapuboids, + GetSchemaPublications(schemaid, true)); } } - puboids = list_concat_unique_oid(puboids, GetAllTablesPublications()); + alltablespuboids = GetAllTablesPublications(); + + /* + * Append "ALL TABLES" publications which does not exclude this + * relation's schema. + */ + puboids = list_concat_unique_oid(puboids, + list_difference_oid(alltablespuboids, + skipschemapuboids)); foreach(lc, puboids) { Oid pubid = lfirst_oid(lc); @@ -5617,7 +5630,8 @@ RelationBuildPublicationDesc(Relation relation, PublicationDesc *pubdesc) if (!pubform->puballtables && (pubform->pubupdate || pubform->pubdelete) && contain_invalid_rfcolumn(pubid, relation, ancestors, - pubform->pubviaroot)) + pubform->pubviaroot, + pubform->puballtables)) { if (pubform->pubupdate) pubdesc->rf_valid_for_update = false; diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c index e5816c4cce..bfe3d8c1ea 100644 --- a/src/bin/pg_dump/pg_dump.c +++ b/src/bin/pg_dump/pg_dump.c @@ -4013,6 +4013,7 @@ getPublicationNamespaces(Archive *fout) int i_oid; int i_pnpubid; int i_pnnspid; + int i_pnskip; int i, j, ntups; @@ -4024,7 +4025,7 @@ getPublicationNamespaces(Archive *fout) /* Collect all publication membership info. */ appendPQExpBufferStr(query, - "SELECT tableoid, oid, pnpubid, pnnspid " + "SELECT tableoid, oid, pnpubid, pnnspid, pnskip " "FROM pg_catalog.pg_publication_namespace"); res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK); @@ -4034,6 +4035,7 @@ getPublicationNamespaces(Archive *fout) i_oid = PQfnumber(res, "oid"); i_pnpubid = PQfnumber(res, "pnpubid"); i_pnnspid = PQfnumber(res, "pnnspid"); + i_pnskip = PQfnumber(res, "pnskip"); /* this allocation may be more than we need */ pubsinfo = pg_malloc(ntups * sizeof(PublicationSchemaInfo)); @@ -4043,6 +4045,7 @@ getPublicationNamespaces(Archive *fout) { Oid pnpubid = atooid(PQgetvalue(res, i, i_pnpubid)); Oid pnnspid = atooid(PQgetvalue(res, i, i_pnnspid)); + char *pnskip = pg_strdup(PQgetvalue(res, i, i_pnskip)); PublicationInfo *pubinfo; NamespaceInfo *nspinfo; @@ -4065,7 +4068,10 @@ getPublicationNamespaces(Archive *fout) continue; /* OK, make a DumpableObject for this relationship */ - pubsinfo[j].dobj.objType = DO_PUBLICATION_TABLE_IN_SCHEMA; + if (strcmp(pnskip, "t") == 0) + pubsinfo[j].dobj.objType = DO_PUBLICATION_SKIP_TABLE_IN_SCHEMA; + else + pubsinfo[j].dobj.objType = DO_PUBLICATION_TABLE_IN_SCHEMA; pubsinfo[j].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid)); pubsinfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid)); @@ -4190,13 +4196,15 @@ getPublicationTables(Archive *fout, TableInfo tblinfo[], int numTables) * dump the definition of the given publication schema mapping. */ static void -dumpPublicationNamespace(Archive *fout, const PublicationSchemaInfo *pubsinfo) +dumpPublicationNamespace(Archive *fout, const PublicationSchemaInfo *pubsinfo, + bool bskip) { DumpOptions *dopt = fout->dopt; NamespaceInfo *schemainfo = pubsinfo->pubschema; PublicationInfo *pubinfo = pubsinfo->publication; PQExpBuffer query; char *tag; + char *description = (bskip) ? "PUBLICATION SKIP TABLES IN SCHEMA" : "PUBLICATION TABLES IN SCHEMA"; /* Do nothing in data-only dump */ if (dopt->dataOnly) @@ -4206,8 +4214,12 @@ dumpPublicationNamespace(Archive *fout, const PublicationSchemaInfo *pubsinfo) query = createPQExpBuffer(); - appendPQExpBuffer(query, "ALTER PUBLICATION %s ", fmtId(pubinfo->dobj.name)); - appendPQExpBuffer(query, "ADD ALL TABLES IN SCHEMA %s;\n", fmtId(schemainfo->dobj.name)); + appendPQExpBuffer(query, "ALTER PUBLICATION %s ADD ", fmtId(pubinfo->dobj.name)); + + if (bskip) + appendPQExpBufferStr(query, "SKIP "); + + appendPQExpBuffer(query, "ALL TABLES IN SCHEMA %s;\n", fmtId(schemainfo->dobj.name)); /* * There is no point in creating drop query as the drop is done by schema @@ -4218,7 +4230,7 @@ dumpPublicationNamespace(Archive *fout, const PublicationSchemaInfo *pubsinfo) ARCHIVE_OPTS(.tag = tag, .namespace = schemainfo->dobj.name, .owner = pubinfo->rolname, - .description = "PUBLICATION TABLES IN SCHEMA", + .description = description, .section = SECTION_POST_DATA, .createStmt = query->data)); @@ -9853,9 +9865,15 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj) case DO_PUBLICATION_REL: dumpPublicationTable(fout, (const PublicationRelInfo *) dobj); break; + case DO_PUBLICATION_SKIP_TABLE_IN_SCHEMA: + dumpPublicationNamespace(fout, + (const PublicationSchemaInfo *) dobj, + true); + break; case DO_PUBLICATION_TABLE_IN_SCHEMA: dumpPublicationNamespace(fout, - (const PublicationSchemaInfo *) dobj); + (const PublicationSchemaInfo *) dobj, + false); break; case DO_SUBSCRIPTION: dumpSubscription(fout, (const SubscriptionInfo *) dobj); @@ -17786,6 +17804,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs, case DO_POLICY: case DO_PUBLICATION: case DO_PUBLICATION_REL: + case DO_PUBLICATION_SKIP_TABLE_IN_SCHEMA: case DO_PUBLICATION_TABLE_IN_SCHEMA: case DO_SUBSCRIPTION: /* Post-data objects: must come after the post-data boundary */ diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h index 772dc0cf7a..2d39975228 100644 --- a/src/bin/pg_dump/pg_dump.h +++ b/src/bin/pg_dump/pg_dump.h @@ -81,6 +81,7 @@ typedef enum DO_POLICY, DO_PUBLICATION, DO_PUBLICATION_REL, + DO_PUBLICATION_SKIP_TABLE_IN_SCHEMA, DO_PUBLICATION_TABLE_IN_SCHEMA, DO_SUBSCRIPTION } DumpableObjectType; diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c index 1592090839..d024936b14 100644 --- a/src/bin/pg_dump/pg_dump_sort.c +++ b/src/bin/pg_dump/pg_dump_sort.c @@ -91,6 +91,7 @@ enum dbObjectTypePriorities PRIO_POLICY, PRIO_PUBLICATION, PRIO_PUBLICATION_REL, + PRIO_PUBLICATION_SKIP_TABLE_IN_SCHEMA, PRIO_PUBLICATION_TABLE_IN_SCHEMA, PRIO_SUBSCRIPTION, PRIO_DEFAULT_ACL, /* done in ACL pass */ @@ -145,6 +146,7 @@ static const int dbObjectTypePriority[] = PRIO_POLICY, /* DO_POLICY */ PRIO_PUBLICATION, /* DO_PUBLICATION */ PRIO_PUBLICATION_REL, /* DO_PUBLICATION_REL */ + PRIO_PUBLICATION_SKIP_TABLE_IN_SCHEMA, /* DO_PUBLICATION_SKIP_TABLE_IN_SCHEMA */ PRIO_PUBLICATION_TABLE_IN_SCHEMA, /* DO_PUBLICATION_TABLE_IN_SCHEMA */ PRIO_SUBSCRIPTION /* DO_SUBSCRIPTION */ }; @@ -1488,6 +1490,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize) "PUBLICATION TABLE (ID %d OID %u)", obj->dumpId, obj->catId.oid); return; + case DO_PUBLICATION_SKIP_TABLE_IN_SCHEMA: + snprintf(buf, bufsize, + "PUBLICATION SKIP TABLES IN SCHEMA (ID %d OID %u)", + obj->dumpId, obj->catId.oid); + return; case DO_PUBLICATION_TABLE_IN_SCHEMA: snprintf(buf, bufsize, "PUBLICATION TABLES IN SCHEMA (ID %d OID %u)", diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl index fd1052e5db..c7b164191e 100644 --- a/src/bin/pg_dump/t/002_pg_dump.pl +++ b/src/bin/pg_dump/t/002_pg_dump.pl @@ -2392,6 +2392,15 @@ my %tests = ( like => { %full_runs, section_post_data => 1, }, }, + 'CREATE PUBLICATION pub5' => { + create_order => 50, + create_sql => 'CREATE PUBLICATION pub5 FOR ALL TABLES;', + regexp => qr/^ + \QCREATE PUBLICATION pub5 FOR ALL TABLES WITH (publish = 'insert, update, delete, truncate');\E + /xm, + like => { %full_runs, section_post_data => 1, }, + }, + 'CREATE SUBSCRIPTION sub1' => { create_order => 50, create_sql => 'CREATE SUBSCRIPTION sub1 @@ -2474,6 +2483,27 @@ my %tests = ( unlike => { exclude_dump_test_schema => 1, }, }, + 'ALTER PUBLICATION pub5 ADD SKIP ALL TABLES IN SCHEMA dump_test' => { + create_order => 51, + create_sql => + 'ALTER PUBLICATION pub5 ADD SKIP ALL TABLES IN SCHEMA dump_test;', + regexp => qr/^ + \QALTER PUBLICATION pub5 ADD SKIP ALL TABLES IN SCHEMA dump_test;\E + /xm, + like => { %full_runs, section_post_data => 1, }, + unlike => { exclude_dump_test_schema => 1, }, + }, + + 'ALTER PUBLICATION pub5 ADD SKIP ALL TABLES IN SCHEMA public' => { + create_order => 52, + create_sql => + 'ALTER PUBLICATION pub5 ADD SKIP ALL TABLES IN SCHEMA public;', + regexp => qr/^ + \QALTER PUBLICATION pub5 ADD SKIP ALL TABLES IN SCHEMA public;\E + /xm, + like => { %full_runs, section_post_data => 1, }, + }, + 'CREATE SCHEMA public' => { regexp => qr/^CREATE SCHEMA public;/m, diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c index 714097cad1..9cba84d694 100644 --- a/src/bin/psql/describe.c +++ b/src/bin/psql/describe.c @@ -2896,6 +2896,7 @@ describeOneTableDetails(const char *schemaname, " JOIN pg_catalog.pg_publication_namespace pn ON p.oid = pn.pnpubid\n" " JOIN pg_catalog.pg_class pc ON pc.relnamespace = pn.pnnspid\n" "WHERE pc.oid ='%s' and pg_catalog.pg_relation_is_publishable('%s')\n" + " AND pn.pnskip = 'f'\n" "UNION\n" "SELECT pubname\n" " , pg_get_expr(pr.prqual, c.oid)\n" @@ -6066,6 +6067,7 @@ describePublications(const char *pattern) "FROM pg_catalog.pg_namespace n\n" " JOIN pg_catalog.pg_publication_namespace pn ON n.oid = pn.pnnspid\n" "WHERE pn.pnpubid = '%s'\n" + " AND pn.pnskip = 'f'\n" "ORDER BY 1", pubid); if (!addFooterToPublicationDesc(&buf, "Tables from schemas:", true, &cont)) @@ -6073,6 +6075,21 @@ describePublications(const char *pattern) } } + if (pset.sversion >= 150000) + { + /* Get the skip schemas for the specified publication */ + printfPQExpBuffer(&buf, + "SELECT n.nspname\n" + "FROM pg_catalog.pg_namespace n\n" + " JOIN pg_catalog.pg_publication_namespace pn ON n.oid = pn.pnnspid\n" + "WHERE pn.pnpubid = '%s'\n" + " AND pn.pnskip = 't'\n" + "ORDER BY 1", pubid); + if (!addFooterToPublicationDesc(&buf, "Skip tables from schemas:", + true, &cont)) + goto error_return; + } + printTable(&cont, pset.queryFout, false, pset.logfile); printTableCleanup(&cont); diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c index 5c064595a9..cdcc33dfb7 100644 --- a/src/bin/psql/tab-complete.c +++ b/src/bin/psql/tab-complete.c @@ -1814,8 +1814,10 @@ psql_completion(const char *text, int start, int end) else if (Matches("ALTER", "PUBLICATION", MatchAny)) COMPLETE_WITH("ADD", "DROP", "OWNER TO", "RENAME TO", "SET"); /* ALTER PUBLICATION <name> ADD */ - else if (Matches("ALTER", "PUBLICATION", MatchAny, "ADD")) - COMPLETE_WITH("ALL TABLES IN SCHEMA", "TABLE"); + else if (Matches("ALTER", "PUBLICATION", MatchAny, "ADD|DROP")) + COMPLETE_WITH("ALL TABLES IN SCHEMA", "SKIP ALL TABLES IN SCHEMA", "TABLE"); + else if (Matches("ALTER", "PUBLICATION", MatchAny, "ADD|DROP", "SKIP")) + COMPLETE_WITH("ALL TABLES IN SCHEMA"); else if (Matches("ALTER", "PUBLICATION", MatchAny, "ADD|SET", "TABLE") || (HeadMatches("ALTER", "PUBLICATION", MatchAny, "ADD|SET", "TABLE") && ends_with(prev_wd, ','))) @@ -1836,13 +1838,13 @@ psql_completion(const char *text, int start, int end) COMPLETE_WITH(",", "WHERE ("); else if (HeadMatches("ALTER", "PUBLICATION", MatchAny, "ADD|SET", "TABLE")) COMPLETE_WITH(","); - /* ALTER PUBLICATION <name> DROP */ - else if (Matches("ALTER", "PUBLICATION", MatchAny, "DROP")) - COMPLETE_WITH("ALL TABLES IN SCHEMA", "TABLE"); /* ALTER PUBLICATION <name> SET */ else if (Matches("ALTER", "PUBLICATION", MatchAny, "SET")) - COMPLETE_WITH("(", "ALL TABLES IN SCHEMA", "TABLE"); - else if (Matches("ALTER", "PUBLICATION", MatchAny, "ADD|DROP|SET", "ALL", "TABLES", "IN", "SCHEMA")) + COMPLETE_WITH("(", "ALL TABLES IN SCHEMA", "SKIP ALL TABLES IN SCHEMA", "TABLE"); + else if (Matches("ALTER", "PUBLICATION", MatchAny, "SET", "SKIP")) + COMPLETE_WITH("ALL TABLES IN SCHEMA"); + else if (Matches("ALTER", "PUBLICATION", MatchAny, "ADD|DROP|SET", "ALL", "TABLES", "IN", "SCHEMA") || + Matches("ALTER", "PUBLICATION", MatchAny, "ADD|DROP|SET", "SKIP", "ALL", "TABLES", "IN", "SCHEMA")) COMPLETE_WITH_QUERY_PLUS(Query_for_list_of_schemas " AND nspname NOT LIKE E'pg\\\\_%%'", "CURRENT_SCHEMA"); @@ -2973,7 +2975,7 @@ psql_completion(const char *text, int start, int end) else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL")) COMPLETE_WITH("TABLES", "TABLES IN SCHEMA"); else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES")) - COMPLETE_WITH("IN SCHEMA", "WITH ("); + COMPLETE_WITH("IN SCHEMA", "SKIP ALL TABLES IN SCHEMA", "WITH ("); else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "TABLE", MatchAny) && !ends_with(prev_wd, ',')) COMPLETE_WITH("WHERE (", "WITH ("); /* Complete "CREATE PUBLICATION <name> FOR TABLE" with "<table>, ..." */ @@ -2995,11 +2997,14 @@ psql_completion(const char *text, int start, int end) * Complete "CREATE PUBLICATION <name> FOR ALL TABLES IN SCHEMA <schema>, * ..." */ - else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "IN", "SCHEMA")) + else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "IN", "SCHEMA") || + Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "SKIP", "ALL", "TABLES", "IN", "SCHEMA")) COMPLETE_WITH_QUERY_PLUS(Query_for_list_of_schemas " AND nspname NOT LIKE E'pg\\\\_%%'", "CURRENT_SCHEMA"); - else if (Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "IN", "SCHEMA", MatchAny) && (!ends_with(prev_wd, ','))) + else if ((Matches("CREATE", "PUBLICATION", MatchAny, "FOR", "ALL", "TABLES", "IN", "SCHEMA", MatchAny) || + Matches("CREATE", "PUBLICATION", MatchAny, "SKIP", "FOR", "ALL", "TABLES", "IN", "SCHEMA", MatchAny)) && + (!ends_with(prev_wd, ','))) COMPLETE_WITH("WITH ("); /* Complete "CREATE PUBLICATION <name> [...] WITH" */ else if (HeadMatches("CREATE", "PUBLICATION") && TailMatches("WITH", "(")) diff --git a/src/include/catalog/pg_publication.h b/src/include/catalog/pg_publication.h index fe773cf9b7..206def7e30 100644 --- a/src/include/catalog/pg_publication.h +++ b/src/include/catalog/pg_publication.h @@ -102,6 +102,13 @@ typedef struct PublicationRelInfo Node *whereClause; } PublicationRelInfo; +typedef struct PublicationSchInfo +{ + NodeTag type; + Oid oid; + bool skip; +} PublicationSchInfo; + extern Publication *GetPublication(Oid pubid); extern Publication *GetPublicationByName(const char *pubname, bool missing_ok); extern List *GetRelationPublications(Oid relid); @@ -124,24 +131,27 @@ typedef enum PublicationPartOpt extern List *GetPublicationRelations(Oid pubid, PublicationPartOpt pub_partopt); extern List *GetAllTablesPublications(void); -extern List *GetAllTablesPublicationRelations(bool pubviaroot); +extern List *GetAllTablesPublicationRelations(Oid pubid, bool pubviaroot); extern List *GetPublicationSchemas(Oid pubid); -extern List *GetSchemaPublications(Oid schemaid); +extern List *GetSchemaPublications(Oid schemaid, bool skippub); extern List *GetSchemaPublicationRelations(Oid schemaid, PublicationPartOpt pub_partopt); extern List *GetAllSchemaPublicationRelations(Oid puboid, - PublicationPartOpt pub_partopt); + PublicationPartOpt pub_partopt, + bool bskip); extern List *GetPubPartitionOptionRelations(List *result, PublicationPartOpt pub_partopt, Oid relid); extern Oid GetTopMostAncestorInPublication(Oid puboid, List *ancestors, - int *ancestor_level); + int *ancestor_level, + bool puballtables); extern bool is_publishable_relation(Relation rel); extern bool is_schema_publication(Oid pubid); extern ObjectAddress publication_add_relation(Oid pubid, PublicationRelInfo *pri, bool if_not_exists); -extern ObjectAddress publication_add_schema(Oid pubid, Oid schemaid, +extern ObjectAddress publication_add_schema(Oid pubid, + PublicationSchInfo *pubsch, bool if_not_exists); extern Oid get_publication_oid(const char *pubname, bool missing_ok); diff --git a/src/include/catalog/pg_publication_namespace.h b/src/include/catalog/pg_publication_namespace.h index e4306da02e..1e1b82cd65 100644 --- a/src/include/catalog/pg_publication_namespace.h +++ b/src/include/catalog/pg_publication_namespace.h @@ -32,6 +32,7 @@ CATALOG(pg_publication_namespace,8901,PublicationNamespaceRelationId) Oid oid; /* oid */ Oid pnpubid BKI_LOOKUP(pg_publication); /* Oid of the publication */ Oid pnnspid BKI_LOOKUP(pg_namespace); /* Oid of the schema */ + bool pnskip BKI_DEFAULT(f); } FormData_pg_publication_namespace; /* ---------------- diff --git a/src/include/commands/publicationcmds.h b/src/include/commands/publicationcmds.h index 7813cbcb6b..0f27cd8347 100644 --- a/src/include/commands/publicationcmds.h +++ b/src/include/commands/publicationcmds.h @@ -32,6 +32,7 @@ extern ObjectAddress AlterPublicationOwner(const char *name, Oid newOwnerId); extern void AlterPublicationOwner_oid(Oid pubid, Oid newOwnerId); extern void InvalidatePublicationRels(List *relids); extern bool contain_invalid_rfcolumn(Oid pubid, Relation relation, - List *ancestors, bool pubviaroot); + List *ancestors, bool pubviaroot, + bool puballtables); #endif /* PUBLICATIONCMDS_H */ diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h index 5d075f0c34..18bba3bf3b 100644 --- a/src/include/nodes/nodes.h +++ b/src/include/nodes/nodes.h @@ -490,6 +490,7 @@ typedef enum NodeTag T_PartitionCmd, T_VacuumRelation, T_PublicationObjSpec, + T_PublicationSchInfo, T_PublicationTable, /* diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 6f83a79a96..1d4cf015fe 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -3672,6 +3672,7 @@ typedef struct PublicationObjSpec PublicationObjSpecType pubobjtype; /* type of this publication object */ char *name; PublicationTable *pubtable; + bool skip; int location; /* token location, or -1 if unknown */ } PublicationObjSpec; diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out index 4e191c120a..faa2ce34e0 100644 --- a/src/test/regress/expected/publication.out +++ b/src/test/regress/expected/publication.out @@ -81,6 +81,35 @@ DETAIL: Tables from schema cannot be added to, dropped from, or set on FOR ALL ALTER PUBLICATION testpub_foralltables SET ALL TABLES IN SCHEMA pub_test; ERROR: publication "testpub_foralltables" is defined as FOR ALL TABLES DETAIL: Tables from schema cannot be added to, dropped from, or set on FOR ALL TABLES publications. +-- should be able to add skip schema to 'FOR ALL TABLES' publication +ALTER PUBLICATION testpub_foralltables ADD SKIP ALL TABLES IN SCHEMA pub_test; +\dRp+ testpub_foralltables + Publication testpub_foralltables + Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root +--------------------------+------------+---------+---------+---------+-----------+---------- + regress_publication_user | t | t | t | f | f | f +Skip tables from schemas: + "pub_test" + +-- should be able to drop skip schema from 'FOR ALL TABLES' publication +ALTER PUBLICATION testpub_foralltables DROP SKIP ALL TABLES IN SCHEMA pub_test; +\dRp+ testpub_foralltables + Publication testpub_foralltables + Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root +--------------------------+------------+---------+---------+---------+-----------+---------- + regress_publication_user | t | t | t | f | f | f +(1 row) + +-- should be able to set skip schema to 'FOR ALL TABLES' publication +ALTER PUBLICATION testpub_foralltables SET SKIP ALL TABLES IN SCHEMA pub_test; +\dRp+ testpub_foralltables + Publication testpub_foralltables + Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root +--------------------------+------------+---------+---------+---------+-----------+---------- + regress_publication_user | t | t | t | f | f | f +Skip tables from schemas: + "pub_test" + SET client_min_messages = 'ERROR'; CREATE PUBLICATION testpub_fortable FOR TABLE testpub_tbl1; RESET client_min_messages; @@ -116,6 +145,18 @@ ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test; Tables from schemas: "pub_test" +-- fail - can't add skip schema to 'FOR TABLE' publication +ALTER PUBLICATION testpub_fortable ADD SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: publication "testpub_fortable" is not defined as FOR ALL TABLES +DETAIL: Skip tables from schema cannot be added to, dropped from, or set on NON ALL TABLES publications. +-- fail - can't drop skip schema from 'FOR TABLE' publication +ALTER PUBLICATION testpub_fortable DROP SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: publication "testpub_fortable" is not defined as FOR ALL TABLES +DETAIL: Skip tables from schema cannot be added to, dropped from, or set on NON ALL TABLES publications. +-- fail - can't set skip schema to 'FOR TABLE' publication +ALTER PUBLICATION testpub_fortable SET SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: publication "testpub_fortable" is not defined as FOR ALL TABLES +DETAIL: Skip tables from schema cannot be added to, dropped from, or set on NON ALL TABLES publications. SET client_min_messages = 'ERROR'; CREATE PUBLICATION testpub_forschema FOR ALL TABLES IN SCHEMA pub_test; RESET client_min_messages; @@ -141,6 +182,18 @@ ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk; Tables: "pub_test.testpub_nopk" +-- fail - can't add skip schema to schema publication +ALTER PUBLICATION testpub_forschema ADD SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: publication "testpub_forschema" is not defined as FOR ALL TABLES +DETAIL: Skip tables from schema cannot be added to, dropped from, or set on NON ALL TABLES publications. +-- fail - can't drop skip schema from schema publication +ALTER PUBLICATION testpub_forschema DROP SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: publication "testpub_forschema" is not defined as FOR ALL TABLES +DETAIL: Skip tables from schema cannot be added to, dropped from, or set on NON ALL TABLES publications. +-- fail - can't set skip schema to schema publication +ALTER PUBLICATION testpub_forschema SET SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: publication "testpub_forschema" is not defined as FOR ALL TABLES +DETAIL: Skip tables from schema cannot be added to, dropped from, or set on NON ALL TABLES publications. SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_foralltables'; pubname | puballtables ----------------------+-------------- @@ -163,10 +216,37 @@ Publications: Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root --------------------------+------------+---------+---------+---------+-----------+---------- regress_publication_user | t | t | t | f | f | f -(1 row) +Skip tables from schemas: + "pub_test" + +SET client_min_messages = 'ERROR'; +CREATE PUBLICATION testpub_foralltables_skipschema FOR ALL TABLES SKIP ALL TABLES IN SCHEMA pub_test; +RESET client_min_messages; +\dRp+ testpub_foralltables_skipschema + Publication testpub_foralltables_skipschema + Owner | All tables | Inserts | Updates | Deletes | Truncates | Via root +--------------------------+------------+---------+---------+---------+-----------+---------- + regress_publication_user | t | t | t | t | t | f +Skip tables from schemas: + "pub_test" +-- fail - can't specify skip schema along with table publication +CREATE PUBLICATION testpub_fortable_skipschema FOR TABLE pub_test.testpub_nopk, SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: SKIP ALL TABLES IN SCHEMA can be specified only with ALL TABLES option +LINE 1: ...E pub_test.testpub_nopk, SKIP ALL TABLES IN SCHEMA pub_test; + ^ +-- fail - can't specify skip schema along with schema publication +CREATE PUBLICATION testpub_forschema_skipschema FOR ALL TABLES IN SCHEMA pub_test, SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: SKIP ALL TABLES IN SCHEMA can be specified only with ALL TABLES option +LINE 1: ...BLES IN SCHEMA pub_test, SKIP ALL TABLES IN SCHEMA pub_test; + ^ +-- fail - can't specify only skip schema while create publication +CREATE PUBLICATION testpub_skipschema FOR SKIP ALL TABLES IN SCHEMA pub_test; +ERROR: SKIP ALL TABLES IN SCHEMA can be specified only with ALL TABLES option +LINE 1: ...N testpub_skipschema FOR SKIP ALL TABLES IN SCHEMA pub_test; + ^ DROP TABLE testpub_tbl2; -DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema; +DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema, testpub_foralltables_skipschema; CREATE TABLE testpub_tbl3 (a int); CREATE TABLE testpub_tbl3a (b text) INHERITS (testpub_tbl3); SET client_min_messages = 'ERROR'; diff --git a/src/test/regress/sql/publication.sql b/src/test/regress/sql/publication.sql index 5457c56b33..de1970f9e0 100644 --- a/src/test/regress/sql/publication.sql +++ b/src/test/regress/sql/publication.sql @@ -58,6 +58,16 @@ ALTER PUBLICATION testpub_foralltables DROP ALL TABLES IN SCHEMA pub_test; -- fail - can't set schema to 'FOR ALL TABLES' publication ALTER PUBLICATION testpub_foralltables SET ALL TABLES IN SCHEMA pub_test; +-- should be able to add skip schema to 'FOR ALL TABLES' publication +ALTER PUBLICATION testpub_foralltables ADD SKIP ALL TABLES IN SCHEMA pub_test; +\dRp+ testpub_foralltables +-- should be able to drop skip schema from 'FOR ALL TABLES' publication +ALTER PUBLICATION testpub_foralltables DROP SKIP ALL TABLES IN SCHEMA pub_test; +\dRp+ testpub_foralltables +-- should be able to set skip schema to 'FOR ALL TABLES' publication +ALTER PUBLICATION testpub_foralltables SET SKIP ALL TABLES IN SCHEMA pub_test; +\dRp+ testpub_foralltables + SET client_min_messages = 'ERROR'; CREATE PUBLICATION testpub_fortable FOR TABLE testpub_tbl1; RESET client_min_messages; @@ -71,6 +81,13 @@ ALTER PUBLICATION testpub_fortable DROP ALL TABLES IN SCHEMA pub_test; ALTER PUBLICATION testpub_fortable SET ALL TABLES IN SCHEMA pub_test; \dRp+ testpub_fortable +-- fail - can't add skip schema to 'FOR TABLE' publication +ALTER PUBLICATION testpub_fortable ADD SKIP ALL TABLES IN SCHEMA pub_test; +-- fail - can't drop skip schema from 'FOR TABLE' publication +ALTER PUBLICATION testpub_fortable DROP SKIP ALL TABLES IN SCHEMA pub_test; +-- fail - can't set skip schema to 'FOR TABLE' publication +ALTER PUBLICATION testpub_fortable SET SKIP ALL TABLES IN SCHEMA pub_test; + SET client_min_messages = 'ERROR'; CREATE PUBLICATION testpub_forschema FOR ALL TABLES IN SCHEMA pub_test; RESET client_min_messages; @@ -85,12 +102,34 @@ ALTER PUBLICATION testpub_forschema DROP TABLE pub_test.testpub_nopk; ALTER PUBLICATION testpub_forschema SET TABLE pub_test.testpub_nopk; \dRp+ testpub_forschema +-- fail - can't add skip schema to schema publication +ALTER PUBLICATION testpub_forschema ADD SKIP ALL TABLES IN SCHEMA pub_test; +-- fail - can't drop skip schema from schema publication +ALTER PUBLICATION testpub_forschema DROP SKIP ALL TABLES IN SCHEMA pub_test; +-- fail - can't set skip schema to schema publication +ALTER PUBLICATION testpub_forschema SET SKIP ALL TABLES IN SCHEMA pub_test; + SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_foralltables'; \d+ testpub_tbl2 \dRp+ testpub_foralltables +SET client_min_messages = 'ERROR'; +CREATE PUBLICATION testpub_foralltables_skipschema FOR ALL TABLES SKIP ALL TABLES IN SCHEMA pub_test; +RESET client_min_messages; + +\dRp+ testpub_foralltables_skipschema + +-- fail - can't specify skip schema along with table publication +CREATE PUBLICATION testpub_fortable_skipschema FOR TABLE pub_test.testpub_nopk, SKIP ALL TABLES IN SCHEMA pub_test; + +-- fail - can't specify skip schema along with schema publication +CREATE PUBLICATION testpub_forschema_skipschema FOR ALL TABLES IN SCHEMA pub_test, SKIP ALL TABLES IN SCHEMA pub_test; + +-- fail - can't specify only skip schema while create publication +CREATE PUBLICATION testpub_skipschema FOR SKIP ALL TABLES IN SCHEMA pub_test; + DROP TABLE testpub_tbl2; -DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema; +DROP PUBLICATION testpub_foralltables, testpub_fortable, testpub_forschema, testpub_foralltables_skipschema; CREATE TABLE testpub_tbl3 (a int); CREATE TABLE testpub_tbl3a (b text) INHERITS (testpub_tbl3); diff --git a/src/test/subscription/t/030_rep_changes_skip_schema.pl b/src/test/subscription/t/030_rep_changes_skip_schema.pl new file mode 100644 index 0000000000..7a16ad350c --- /dev/null +++ b/src/test/subscription/t/030_rep_changes_skip_schema.pl @@ -0,0 +1,96 @@ + +# Copyright (c) 2021-2022, PostgreSQL Global Development Group + +# Logical replication tests for skip schema publications +use strict; +use warnings; +use PostgreSQL::Test::Cluster; +use PostgreSQL::Test::Utils; +use Test::More; + +# Initialize publisher node +my $node_publisher = PostgreSQL::Test::Cluster->new('publisher'); +$node_publisher->init(allows_streaming => 'logical'); +$node_publisher->start; + +# Create subscriber node +my $node_subscriber = PostgreSQL::Test::Cluster->new('subscriber'); +$node_subscriber->init(allows_streaming => 'logical'); +$node_subscriber->start; + +# Test replication with publications created using FOR ALL TABLES SKIP ALL TABLES IN SCHEMA +# option. +# Create schemas and tables on publisher +$node_publisher->safe_psql('postgres', "CREATE SCHEMA sch1"); +$node_publisher->safe_psql('postgres', + "CREATE TABLE sch1.tab1 AS SELECT generate_series(1,10) AS a"); +$node_publisher->safe_psql('postgres', + "CREATE TABLE public.tab1(a int)"); + +# Create schemas and tables on subscriber +$node_subscriber->safe_psql('postgres', "CREATE SCHEMA sch1"); +$node_subscriber->safe_psql('postgres', "CREATE TABLE sch1.tab1 (a int)"); +$node_subscriber->safe_psql('postgres', "CREATE TABLE public.tab1 (a int)"); + +# Setup logical replication +my $publisher_connstr = $node_publisher->connstr . ' dbname=postgres'; +$node_publisher->safe_psql('postgres', + "CREATE PUBLICATION tap_pub_schema FOR ALL TABLES SKIP ALL TABLES IN SCHEMA sch1"); + +$node_subscriber->safe_psql('postgres', + "CREATE SUBSCRIPTION tap_sub_schema CONNECTION '$publisher_connstr' PUBLICATION tap_pub_schema" +); + +$node_publisher->wait_for_catchup('tap_sub_schema'); + +# Also wait for initial table sync to finish +my $synced_query = + "SELECT count(1) = 0 FROM pg_subscription_rel WHERE srsubstate NOT IN ('r', 's');"; +$node_subscriber->poll_query_until('postgres', $synced_query) + or die "Timed out while waiting for subscriber to synchronize data"; + +# Check the schema table data does not sync for skip schemas +my $result = $node_subscriber->safe_psql('postgres', + "SELECT count(*), min(a), max(a) FROM sch1.tab1"); +is($result, qq(0||), 'check tablesync is skipped for skip schemas'); + +# Insert some data into few tables and verify that inserted data is not replicated +$node_publisher->safe_psql('postgres', + "INSERT INTO sch1.tab1 VALUES(generate_series(11,20))"); + +$node_publisher->wait_for_catchup('tap_sub_schema'); + +$result = $node_subscriber->safe_psql('postgres', + "SELECT count(*), min(a), max(a) FROM sch1.tab1"); +is($result, qq(0||), 'check replicated inserts on subscriber'); + +# Alter publication to skip data changes in public and verify that subscriber does not get +# the new table data. +$node_publisher->safe_psql('postgres', + "ALTER PUBLICATION tap_pub_schema add SKIP ALL TABLES IN SCHEMA public"); +$node_publisher->safe_psql('postgres', + "INSERT INTO public.tab1 VALUES(generate_series(1,10))"); + +$node_publisher->wait_for_catchup('tap_sub_schema'); + +$result = $node_subscriber->safe_psql('postgres', + "SELECT count(*), min(a), max(a) FROM public.tab1"); +is($result, qq(0||), 'check rows on subscriber catchup'); + +# Alter publication to drop skip schema public and verify that subscriber gets +# the new table data. +$node_publisher->safe_psql('postgres', + "ALTER PUBLICATION tap_pub_schema drop SKIP ALL TABLES IN SCHEMA public"); +$node_publisher->safe_psql('postgres', + "INSERT INTO public.tab1 VALUES(generate_series(1,10))"); + +$node_publisher->wait_for_catchup('tap_sub_schema'); + +$result = $node_subscriber->safe_psql('postgres', + "SELECT count(*), min(a), max(a) FROM public.tab1"); +is($result, qq(10|1|10), 'check rows on subscriber catchup'); + +$node_subscriber->stop('fast'); +$node_publisher->stop('fast'); + +done_testing(); diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 93d5190508..e8569f2835 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -2065,6 +2065,7 @@ PublicationObjSpecType PublicationPartOpt PublicationRelInfo PublicationSchemaInfo +PublicationSchInfo PublicationTable PullFilter PullFilterOps -- 2.32.0