On Fri Oct 2, 2026 at 1:30 AM CEST, Alberto Piai wrote: > On Thu Sep 24, 2026 at 12:47 PM CEST, Laurenz Albe wrote: >> On Thu, 2026-09-24 at 00:19 +0200, Alberto Piai wrote: >>> I find Matthias' proposal of exposing a function to check image equality >>> very compelling for the purpose of this patch: besides fixing this >>> problem, it would also make the command usable for data types which >>> don't define = (json), as well as those which don't (can't?) define >>> equalimage()... jsonb, numeric but also tsvector and PostGIS geometry. >> >> True, "tsvector" is limiting; I can see people wanting that for >> generated columns.
I propose to move forward with pg_datum_image_equal() as the contract for the check constraint. It is simpler for the user. We can always reconsider the ROW()::record indirection in case the datum image equality patch cannot be committed for some reason. So here's v12, rebased on top of master + Matthias' pg_datum_image_equal() patch from [0]. It changes the implementation to require the new constraint shape, and updates all the documentation, tests and error messages. Besides that, I made a few more changes since v11: - bail if the column is part of a FK with on delete/update actions: adding the column directly in CREATE TABLE and ALTER TABLE ADD COLUMN also rejects that. - clear atthasmissing and attmissingval for consistency with CREATE TABLE: a column that was GENERATED since the beginning can't have those set, so let's be consistent. - remove support for foreign tables: I don't see the point of supporting this, since adding a stored generated column on a foreign table does not involve a rewrite nor a scan. The actual data migration would have to be done on the remote anyway. - fix wrong type for a pg_depend.refobjsubid lookup. - release AccessShareLock after scanning pg_depend: it's inconsequential since the target rel is already under AccessExclusiveLock, so no concurrent DDL involving it would happen. But it makes the function more self contained. - fix detection of trivial constraint col1=col2 (two Vars, but in swapped order). - add a tsvector test, also for dump/restore. - a few minor fixes, such as an incorrect use of errmsg instead of errmsg_internal and an incorrect comment. Looking forward to your thoughts on this! Regards, Alberto [0] https://postgr.es/m/caeze2whw41mvdmkrcplaj_nmlvp4jtqlzjtg3m8rxnnbhzm...@mail.gmail.com -- Alberto Piai Sensational AG Zürich, Switzerland
>From 08b055fc84f70715ae8a6bfb155630ede56f3d7a Mon Sep 17 00:00:00 2001 From: Matthias van de Meent <[email protected]> Date: Wed, 10 Dec 2025 18:45:19 +0100 Subject: [PATCH v12 1/2] Add SQL-level datum equality tests This enables improved performance for users that need to test for exact bytewise differences in SQL; e.g. to see if an UPDATE is really necessary with a set of externally provided values. A workaround around this limitation was often possible, but no generic method was available that was this performant, nor as accessible to all types. --- doc/src/sgml/func/func-comparison.sgml | 25 ++++++ src/backend/utils/adt/pseudotypes.c | 51 ++++++++++++ src/include/catalog/pg_proc.dat | 7 ++ src/test/regress/expected/misc_functions.out | 81 ++++++++++++++++++++ src/test/regress/expected/opr_sanity.out | 1 + src/test/regress/sql/misc_functions.sql | 25 ++++++ 6 files changed, 190 insertions(+) diff --git a/doc/src/sgml/func/func-comparison.sgml b/doc/src/sgml/func/func-comparison.sgml index ecb1d89463a..df806d992c3 100644 --- a/doc/src/sgml/func/func-comparison.sgml +++ b/doc/src/sgml/func/func-comparison.sgml @@ -653,6 +653,31 @@ SELECT NOT(ROW(table.*) IS NOT NULL) FROM TABLE; -- detect at least one null in <returnvalue>1</returnvalue> </para></entry> </row> + + <row> + <entry role="func_table_entry"><para role="func_signature"> + <indexterm> + <primary>pg_datum_image_equal</primary> + </indexterm> + <function>pg_datum_image_equal</function> ( <type>anyelement</type>, <type>anyelement</type> ) + <returnvalue>boolean</returnvalue> + </para> + <para> + Returns whether the values have the same exact binary representation. + </para> + <para> + <literal>pg_datum_image_equal('1.0'::numeric, '1.0'::numeric)</literal> + <returnvalue>true</returnvalue> + </para> + <para> + <literal>pg_datum_image_equal('1.0'::numeric, '1.00'::numeric)</literal> + <returnvalue>false</returnvalue> + </para> + <para> + <literal>pg_datum_image_equal(NULL::numeric, NULL::numeric)</literal> + <returnvalue>true</returnvalue> + </para></entry> + </row> </tbody> </tgroup> </table> diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c index 4581c4b1697..93a67e53026 100644 --- a/src/backend/utils/adt/pseudotypes.c +++ b/src/backend/utils/adt/pseudotypes.c @@ -23,7 +23,9 @@ #include "postgres.h" #include "libpq/pqformat.h" +#include "utils/datum.h" #include "utils/fmgrprotos.h" +#include "utils/lsyscache.h" /* @@ -375,3 +377,52 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement); PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray); PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible); PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray); + +/* + * Compares two datums of the same (any) type, and returns whether they have + * the same binary representation. + */ +Datum +pg_datum_image_equal(PG_FUNCTION_ARGS) +{ + bool eq; + + if (PG_ARGISNULL(0) != PG_ARGISNULL(1)) + { + eq = false; + } + else if (PG_ARGISNULL(0)) + { + /* both NULL */ + eq = true; + } + else + { + Oid typ; + Datum arg0; + Datum arg1; + bool typbyval; + char typalign; + int16 typlen; + + typ = get_fn_expr_argtype(fcinfo->flinfo, 0); + + if (!OidIsValid(typ)) + { + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("could not determine type"))); + } + + Assert(typ == get_fn_expr_argtype(fcinfo->flinfo, 1)); + + arg0 = PG_GETARG_DATUM(0); + arg1 = PG_GETARG_DATUM(1); + + get_typlenbyvalalign(typ, &typlen, &typbyval, &typalign); + + eq = datum_image_eq(arg0, arg1, typbyval, typlen); + } + + PG_RETURN_BOOL(eq); +} diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat index 9b78fcad414..7feb1015cf8 100644 --- a/src/include/catalog/pg_proc.dat +++ b/src/include/catalog/pg_proc.dat @@ -12783,4 +12783,11 @@ proname => 'hashoid8extended', prorettype => 'int8', proargtypes => 'oid8 int8', prosrc => 'hashoid8extended' }, +{ oid => '9200', + descr => 'test if two values have the same binary representation', + proname => 'pg_datum_image_equal', proisstrict => 'f', + proleakproof => 't', prorettype => 'bool', + proargtypes => 'anyelement anyelement', + prosrc => 'pg_datum_image_equal' }, + ] diff --git a/src/test/regress/expected/misc_functions.out b/src/test/regress/expected/misc_functions.out index 35d6ee2ce9e..7976ad8e925 100644 --- a/src/test/regress/expected/misc_functions.out +++ b/src/test/regress/expected/misc_functions.out @@ -901,3 +901,84 @@ SELECT test_pg_locale_apis(to_regcollation('en-x-icu')); (1 row) +-- test pg_datum_image_equal(..., ...) +WITH values AS ( + SELECT val::numeric FROM ( + VALUES ('1.0'), + ('1.00'), + ('2.0'), + ('2.1'), + (NULL) + ) AS v(val) +) +SELECT a.val a, b.val b, a.val = b.val eq, pg_datum_image_equal(a.val, b.val) pdie +FROM values a CROSS JOIN values b; + a | b | eq | pdie +------+------+----+------ + 1.0 | 1.0 | t | t + 1.0 | 1.00 | t | f + 1.0 | 2.0 | f | f + 1.0 | 2.1 | f | f + 1.0 | | | f + 1.00 | 1.0 | t | f + 1.00 | 1.00 | t | t + 1.00 | 2.0 | f | f + 1.00 | 2.1 | f | f + 1.00 | | | f + 2.0 | 1.0 | f | f + 2.0 | 1.00 | f | f + 2.0 | 2.0 | t | t + 2.0 | 2.1 | f | f + 2.0 | | | f + 2.1 | 1.0 | f | f + 2.1 | 1.00 | f | f + 2.1 | 2.0 | f | f + 2.1 | 2.1 | t | t + 2.1 | | | f + | 1.0 | | f + | 1.00 | | f + | 2.0 | | f + | 2.1 | | f + | | | t +(25 rows) + +WITH values AS ( + SELECT val::jsonb FROM ( + VALUES ('{"key": 1.0}'), + ('{"key": 1.00}'), + ('{"key": 2.0}'), + ('{"key": null}'), + (NULL) + ) AS v(val) +) +SELECT a.val a, b.val b, a.val = b.val eq, pg_datum_image_equal(a.val, b.val) pdie +FROM values a CROSS JOIN values b; + a | b | eq | pdie +---------------+---------------+----+------ + {"key": 1.0} | {"key": 1.0} | t | t + {"key": 1.0} | {"key": 1.00} | t | f + {"key": 1.0} | {"key": 2.0} | f | f + {"key": 1.0} | {"key": null} | f | f + {"key": 1.0} | | | f + {"key": 1.00} | {"key": 1.0} | t | f + {"key": 1.00} | {"key": 1.00} | t | t + {"key": 1.00} | {"key": 2.0} | f | f + {"key": 1.00} | {"key": null} | f | f + {"key": 1.00} | | | f + {"key": 2.0} | {"key": 1.0} | f | f + {"key": 2.0} | {"key": 1.00} | f | f + {"key": 2.0} | {"key": 2.0} | t | t + {"key": 2.0} | {"key": null} | f | f + {"key": 2.0} | | | f + {"key": null} | {"key": 1.0} | f | f + {"key": null} | {"key": 1.00} | f | f + {"key": null} | {"key": 2.0} | f | f + {"key": null} | {"key": null} | t | t + {"key": null} | | | f + | {"key": 1.0} | | f + | {"key": 1.00} | | f + | {"key": 2.0} | | f + | {"key": null} | | f + | | | t +(25 rows) + diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out index 907d63daa3f..281c46ff6ec 100644 --- a/src/test/regress/expected/opr_sanity.out +++ b/src/test/regress/expected/opr_sanity.out @@ -892,6 +892,7 @@ tid_block(tid) tid_offset(tid) uuid_larger(uuid,uuid) uuid_smaller(uuid,uuid) +pg_datum_image_equal(anyelement,anyelement) -- Check that functions without argument are not marked as leakproof. SELECT p1.oid::regprocedure FROM pg_proc p1 JOIN pg_namespace pn diff --git a/src/test/regress/sql/misc_functions.sql b/src/test/regress/sql/misc_functions.sql index 94b91964c04..e509b05c658 100644 --- a/src/test/regress/sql/misc_functions.sql +++ b/src/test/regress/sql/misc_functions.sql @@ -381,3 +381,28 @@ SELECT test_pg_locale_apis(to_regcollation('pg_c_utf8')); -- en-x-icu is present when ICU collations were imported at initdb. SELECT test_pg_locale_apis(to_regcollation('en-x-icu')); + +-- test pg_datum_image_equal(..., ...) +WITH values AS ( + SELECT val::numeric FROM ( + VALUES ('1.0'), + ('1.00'), + ('2.0'), + ('2.1'), + (NULL) + ) AS v(val) +) +SELECT a.val a, b.val b, a.val = b.val eq, pg_datum_image_equal(a.val, b.val) pdie +FROM values a CROSS JOIN values b; + +WITH values AS ( + SELECT val::jsonb FROM ( + VALUES ('{"key": 1.0}'), + ('{"key": 1.00}'), + ('{"key": 2.0}'), + ('{"key": null}'), + (NULL) + ) AS v(val) +) +SELECT a.val a, b.val b, a.val = b.val eq, pg_datum_image_equal(a.val, b.val) pdie +FROM values a CROSS JOIN values b; base-commit: 4a934833fb02ac2a18d0a3f86b67b493d7fe8979 -- 2.54.0
>From 1586a9864fb37fc48934de819fd62d8038c61326 Mon Sep 17 00:00:00 2001 From: Alberto Piai <[email protected]> Date: Fri, 6 Nov 2026 13:12:28 +0100 Subject: [PATCH v12 2/2] Support changing a column into a stored generated column This adds an ALTER TABLE subcommand which turns a regular column into a stored generated column: ... ALTER col ADD GENERATED USING CONSTRAINT constr_name STORED The main purpose of this command is to make it possible to add a stored generated column without rewriting the table under an AccessExclusive lock. Before running this command, the table should have been prepared by adding a regular column, backfilling it with data matching the intended generation expression, and adding the constr_name constraint to prove that the data does satisfy said generation expression. The constraint must have a specific structure to be usable for this operation: CHECK (pg_datum_image_equal(column_name, expr)) The column will then be changed into a stored generated column, with the "expr" from the constraint as its generator expression. The operation will be performed without rewriting the table, and without any verification scan. The syntax is chosen for its similarity to: ... ALTER COLUMN ... ADD GENERATED ... AS IDENTITY with the difference that: - since we are not talking about IDENTITY columns, ALWAYS is implicit and does not need to be explicitly supplied - STORED must always be supplied, to make it clear that we are talking about stored and not virtual generated columns (VIRTUAL is the default when omitted in other commands, so we mandate STORED here to be consistent) The use of the pg_datum_image_equal() function in the constraint is necessary to ensure that the data in the column is exactly what would be produced by the generator expression: if we used logical = instead, it would be possible to store data inconsistent with the generator expression by using values which are equal according to =, but don't have the same image (for example numeric 1.0 and 1.00). This new operation fits together with SET EXPRESSION and DROP EXPRESSION: the latter works in the opposite direction, turning a generated column into a regular column. --- doc/src/sgml/ref/alter_table.sgml | 36 ++ src/backend/commands/tablecmds.c | 565 +++++++++++++++++- src/backend/parser/gram.y | 18 + src/backend/utils/adt/pseudotypes.c | 12 +- src/bin/psql/t/010_tab_completion.pl | 22 +- src/bin/psql/tab-complete.in.c | 43 +- src/include/nodes/parsenodes.h | 1 + src/test/modules/injection_points/Makefile | 2 +- .../injection_points/expected/alter_table.out | 36 ++ src/test/modules/injection_points/meson.build | 1 + .../injection_points/sql/alter_table.sql | 24 + .../test_ddl_deparse/test_ddl_deparse.c | 3 + src/test/regress/expected/alter_table.out | 458 ++++++++++++++ src/test/regress/sql/alter_table.sql | 290 +++++++++ src/tools/pgindent/typedefs.list | 1 + 15 files changed, 1499 insertions(+), 13 deletions(-) create mode 100644 src/test/modules/injection_points/expected/alter_table.out create mode 100644 src/test/modules/injection_points/sql/alter_table.sql diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml index 0f9d698d170..44afaf5f4f6 100644 --- a/doc/src/sgml/ref/alter_table.sgml +++ b/doc/src/sgml/ref/alter_table.sgml @@ -46,6 +46,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET DEFAULT <replaceable class="parameter">expression</replaceable> ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP DEFAULT ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> { SET | DROP } NOT NULL + ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED USING CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> STORED ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET EXPRESSION AS ( <replaceable class="parameter">expression</replaceable> ) ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> DROP EXPRESSION [ IF EXISTS ] ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> ADD GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] @@ -266,6 +267,41 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM </listitem> </varlistentry> + <varlistentry id="sql-altertable-desc-add-generated-always-stored-using-constraint"> + <term><literal>ADD GENERATED USING CONSTRAINT</literal></term> + <listitem> + <para> + This form changes a regular column into a stored generated column, using + the expression from the given <literal>CHECK</literal> constraint as + generation expression. The operation will be performed without rewriting + the table, which avoids holding an <literal>ACCESS EXCLUSIVE</literal> + lock for a possibly long time. + </para> + + <para> + Before using this command, the table will usually have been prepared by + adding a regular column, backfilling it with values matching the intended + generation expression and adding a constraint to ensure that the + generation expression is satisfied. Note that the constraint can also be + added without holding an <literal>ACCESS EXCLUSIVE</literal> lock while + the table is scanned, using <literal>NOT VALID</literal> and + <literal>VALIDATE CONSTRAINT</literal>. + </para> + + <para> + The constraint must be of the form <literal>CHECK ( pg_datum_image_equal( + <replaceable class="parameter">column_name</replaceable>, + <replaceable class="parameter">expr</replaceable>) )</literal>. + </para> + + <para> + After this command has completed, <literal>column_name</literal> will be a + stored generated column with <literal>expr</literal> as its generation + expression. + </para> + </listitem> + </varlistentry> + <varlistentry id="sql-altertable-desc-set-expression"> <term><literal>SET EXPRESSION AS</literal></term> <listitem> diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index 47f55121574..c0f9dae622a 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -99,6 +99,7 @@ #include "utils/acl.h" #include "utils/builtins.h" #include "utils/fmgroids.h" +#include "utils/injection_point.h" #include "utils/inval.h" #include "utils/lsyscache.h" #include "utils/memutils.h" @@ -327,6 +328,17 @@ struct DropRelationCallbackState char actual_relpersistence; }; +/* + * Used by findUsableConstraintForAddGenStored to give a hint about why no + * constraint matched. + */ +typedef enum AddGenConstrError +{ + ADD_GEN_CONSTR_NOT_FOUND = 0, + ADD_GEN_CONSTR_NOT_VALID, + ADD_GEN_CONSTR_SHAPE_MISMATCH, +} AddGenConstrError; + /* Alter table target-type flags for ATSimplePermissions */ #define ATT_TABLE 0x0001 #define ATT_VIEW 0x0002 @@ -756,7 +768,20 @@ static List *GetParentedForeignKeyRefs(Relation partition); static void ATDetachCheckNoForeignKeyRefs(Relation partition); static char GetAttributeCompression(Oid atttypid, const char *compression); static char GetAttributeStorage(Oid atttypid, const char *storagemode); - +static void ATPrepAddGenStored(Relation rel, AlterTableCmd *cmd, bool recurse, bool recursing); +static void checkDependenciesForAddGenStored(Relation rel, + AttrNumber attnum, + const char *colName); +static Node *findUsableConstraintForAddGenStored(Relation rel, + AttrNumber attnum, + const char *conname, + AddGenConstrError *reason); +static Node *matchDatumImageEqualOnVar(Node *expr, AttrNumber attnum); +static Node *reconstructRawExpr(Relation rel, Node *cookedExpr); +static ObjectAddress ATExecAddGeneratedStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def); /* ---------------------------------------------------------------- * DefineRelation @@ -4770,6 +4795,7 @@ AlterTableGetLockLevel(List *cmds) case AT_AddIdentity: case AT_DropIdentity: case AT_SetIdentity: + case AT_AddGeneratedStored: case AT_SetExpression: case AT_DropExpression: case AT_SetCompression: @@ -5089,6 +5115,14 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd, ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); pass = AT_PASS_SET_EXPRESSION; break; + case AT_AddGeneratedStored: /* ALTER COLUMN ADD GENERATED USING + * CONSTRAINT */ + ATSimplePermissions(cmd->subtype, rel, + ATT_TABLE | ATT_PARTITIONED_TABLE); + ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context); + ATPrepAddGenStored(rel, cmd, recurse, recursing); + pass = AT_PASS_ADD_OTHERCONSTR; + break; case AT_DropExpression: /* ALTER COLUMN DROP EXPRESSION */ ATSimplePermissions(cmd->subtype, rel, ATT_TABLE | ATT_PARTITIONED_TABLE | ATT_FOREIGN_TABLE); @@ -5489,6 +5523,12 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, case AT_SetExpression: address = ATExecSetExpression(tab, rel, cmd->name, cmd->def, lockmode); break; + case AT_AddGeneratedStored: + Assert(IsA(cmd->def, Constraint)); + address = ATExecAddGeneratedStored(tab, rel, + cmd->name, + (Constraint *) cmd->def); + break; case AT_DropExpression: address = ATExecDropExpression(rel, cmd->name, cmd->missing_ok, lockmode); break; @@ -6355,13 +6395,19 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap) } if (newrel) + { ereport(DEBUG1, (errmsg_internal("rewriting table \"%s\"", RelationGetRelationName(oldrel)))); + INJECTION_POINT("alter-table-phase-3-rewrite", NULL); + } else + { ereport(DEBUG1, (errmsg_internal("verifying table \"%s\"", RelationGetRelationName(oldrel)))); + INJECTION_POINT("alter-table-phase-3-verify", NULL); + } if (newrel) { @@ -6681,6 +6727,8 @@ alter_table_type_to_string(AlterTableType cmdtype) return "ALTER COLUMN ... SET NOT NULL"; case AT_SetExpression: return "ALTER COLUMN ... SET EXPRESSION"; + case AT_AddGeneratedStored: + return "ALTER COLUMN ... ADD GENERATED STORED"; case AT_DropExpression: return "ALTER COLUMN ... DROP EXPRESSION"; case AT_SetStatistics: @@ -8836,6 +8884,521 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, return address; } +/* + * Preparation phase for + * + * ALTER COLUMN col ADD GENERATED USING CONSTRAINT name STORED + * + * In an inheritance hierarchy, it is only valid to alter the type of the + * whole hierarchy at once. + */ +static void +ATPrepAddGenStored(Relation rel, AlterTableCmd *cmd, bool recurse, bool recursing) +{ + /* + * This routine is called on the top table directly with recursing=false, + * and on all children tables via ATSimpleRecursion with recursing=true. + * + * At the top level, forbid ONLY (i.e. recurse=false) if there are child + * tables. We only check this at the top level, otherwise we would prevent + * this operation from being applied to hierarchies with depth > 2. + * + * The caller already holds an AccessExclusiveLock on the rel, so taking + * locks in find_inheritance_children is not necessary. + */ + if (!recursing && !recurse && + find_inheritance_children(RelationGetRelid(rel), NoLock)) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot convert column \"%s\" to generated", cmd->name), + errdetail("Converting only part of a partitioning or inheritance hierarchy is not supported."), + errhint("Use this command on the root table without specifying ONLY.")); + + /* + * Don't allow this operation to be applied to inherited columns directly. + */ + if (!recursing) + { + HeapTuple tuple; + Form_pg_attribute attTup; + + tuple = SearchSysCacheCopyAttName(RelationGetRelid(rel), cmd->name); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + cmd->name, RelationGetRelationName(rel))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + if (attTup->attinhcount > 0) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot convert column \"%s\" to generated", cmd->name), + errdetail("Converting only part of a partitioning or inheritance hierarchy is not supported."), + errhint("Use this command on the root table without specifying ONLY.")); + } +} + +/* + * Detect dependencies which should stop us from turning a regular column + * into a stored generated column. + */ +static void +checkDependenciesForAddGenStored(Relation rel, + AttrNumber attnum, + const char *colName) +{ + Relation pg_depend; + ScanKeyData key[3]; + SysScanDesc scan; + HeapTuple depTup; + + pg_depend = table_open(DependRelationId, AccessShareLock); + + ScanKeyInit(&key[0], + Anum_pg_depend_refclassid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationRelationId)); + ScanKeyInit(&key[1], + Anum_pg_depend_refobjid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(RelationGetRelid(rel))); + ScanKeyInit(&key[2], + Anum_pg_depend_refobjsubid, + BTEqualStrategyNumber, F_INT4EQ, + Int32GetDatum(attnum)); + + scan = systable_beginscan(pg_depend, DependReferenceIndexId, true, + NULL, 3, key); + + while (HeapTupleIsValid(depTup = systable_getnext(scan))) + { + Form_pg_depend dep = GETSTRUCT(depTup); + ObjectAddress foundObject; + + foundObject.classId = dep->classid; + foundObject.objectId = dep->objid; + foundObject.objectSubId = dep->objsubid; + + switch (foundObject.classId) + { + case RelationRelationId: + { + char relKind = get_rel_relkind(foundObject.objectId); + + /* + * While it is possible to alter any sequence to be owned + * by an arbitrary column, the most likely legitimate use + * is for a serial column. Let's assume this is the case + * for the sake of a more helpful error message. + */ + if (relKind == RELKIND_SEQUENCE) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("Column \"%s\" of relation \"%s\" is a serial column (depends on \"%s\").", + colName, RelationGetRelationName(rel), + getObjectDescription(&foundObject, false)))); + break; + } + case AttrDefaultRelationId: + { + ObjectAddress col = GetAttrDefaultColumnAddress(foundObject.objectId); + + if (col.objectId == RelationGetRelid(rel) && + col.objectSubId == attnum) + { + /* + * Ignore the column's own default expression. We + * handle sequences above, and for a column which is + * already a generated column we should never get + * here. + */ + } + else + { + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("Column \"%s\" is referenced by generated column \"%s\".", + colName, + get_attname(col.objectId, col.objectSubId, false)))); + } + break; + } + case ConstraintRelationId: + { + if (get_constraint_type(foundObject.objectId) == CONSTRAINT_FOREIGN) + { + HeapTuple conTup; + Form_pg_constraint con; + + conTup = SearchSysCache1(CONSTROID, ObjectIdGetDatum(foundObject.objectId)); + if (!HeapTupleIsValid(conTup)) + elog(ERROR, "cache lookup failed for constraint %u", foundObject.objectId); + con = (Form_pg_constraint) GETSTRUCT(conTup); + + if (con->confupdtype == FKCONSTR_ACTION_SETNULL || + con->confupdtype == FKCONSTR_ACTION_SETDEFAULT || + con->confupdtype == FKCONSTR_ACTION_CASCADE || + con->confdeltype == FKCONSTR_ACTION_SETNULL || + con->confdeltype == FKCONSTR_ACTION_SETDEFAULT) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("Column \"%s\" is part of a foreign key with an ON DELETE / ON UPDATE action.", + colName))); + + ReleaseSysCache(conTup); + } + break; + } + default: + /* Other dependencies aren't a problem. */ + break; + } + } + + systable_endscan(scan); + table_close(pg_depend, AccessShareLock); +} + +/* + * Subroutine for ATExecAddGeneratedStored, used to determine whether the given + * constraint proves that the values are equal to some expression. + * + * Given a rel, a column and a constraint name, we look up a valid CHECK + * constraint on the rel, with the given name, with a specific shape: + * + * CHECK ( pg_datum_image_equal(column, expr) ) + * + * If a valid constraint is found, this returns the expr node, otherwise + * it returns null and sets an error code in *reason, allowing the caller to + * provide an appropriate error message. + */ +static Node * +findUsableConstraintForAddGenStored(Relation rel, AttrNumber attnum, + const char *conname, + AddGenConstrError *reason) +{ + Relation pg_constraint; + HeapTuple conTup; + SysScanDesc scan; + ScanKeyData key[3]; + Node *foundExpr; + + pg_constraint = table_open(ConstraintRelationId, AccessShareLock); + ScanKeyInit(&key[0], + Anum_pg_constraint_conrelid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(rel->rd_id)); + ScanKeyInit(&key[1], + Anum_pg_constraint_contypid, + BTEqualStrategyNumber, F_OIDEQ, + ObjectIdGetDatum(InvalidOid)); + ScanKeyInit(&key[2], + Anum_pg_constraint_conname, + BTEqualStrategyNumber, F_NAMEEQ, + CStringGetDatum(conname)); + scan = systable_beginscan(pg_constraint, ConstraintRelidTypidNameIndexId, + true, NULL, 3, key); + + foundExpr = NULL; + *reason = ADD_GEN_CONSTR_NOT_FOUND; + + if (HeapTupleIsValid(conTup = systable_getnext(scan))) + { + Form_pg_constraint con = GETSTRUCT(conTup); + + if (con->contype == CONSTRAINT_CHECK) + { + /* !conenforced implies !convalidated, but let's be explicit */ + if (!con->convalidated || !con->conenforced) + { + *reason = ADD_GEN_CONSTR_NOT_VALID; + } + else + { + Datum val; + char *conbin; + Node *conexpr; + + val = SysCacheGetAttrNotNull(CONSTROID, conTup, + Anum_pg_constraint_conbin); + conbin = TextDatumGetCString(val); + conexpr = stringToNode(conbin); + + foundExpr = matchDatumImageEqualOnVar(conexpr, attnum); + if (foundExpr == NULL) + *reason = ADD_GEN_CONSTR_SHAPE_MISMATCH; + } + } + } + + systable_endscan(scan); + table_close(pg_constraint, AccessShareLock); + + return foundExpr; +} + +/* + * Given an expression tree and an attnum, this tries to match + * + * pg_datum_image_equal(arg1, arg2) + * + * where either arg1 or arg2 is a Var referencing the given attno, returning + * the other argument. + */ +static Node * +matchDatumImageEqualOnVar(Node *expr, AttrNumber attnum) +{ + if (IsA(expr, FuncExpr)) + { + FuncExpr *fexpr = (FuncExpr *) expr; + + if (fexpr->funcid == F_PG_DATUM_IMAGE_EQUAL && + list_length(fexpr->args) == 2) + { + Node *left = linitial(fexpr->args), + *right = lsecond(fexpr->args); + + /* Support both orders of the operands */ + if (IsA(left, Var) && ((Var *) left)->varattno == attnum) + { + Assert(((Var *) left)->vartype == exprType(right)); + return right; + } + if (IsA(right, Var) && ((Var *) right)->varattno == attnum) + { + Assert(((Var *) right)->vartype == exprType(left)); + return left; + } + } + } + + return NULL; +} + +/* + * Reconstruct a raw expression from a given cooked expression by deparsing it + * and running it through raw_parser(). + */ +static Node * +reconstructRawExpr(Relation rel, Node *cookedExpr) +{ + char *deparsedExpr; + List *ctx, + *parseResult = NIL; + + ctx = deparse_context_for(RelationGetRelationName(rel), + RelationGetRelid(rel)); + + deparsedExpr = deparse_expression(cookedExpr, ctx, false, false); + + parseResult = raw_parser(deparsedExpr, RAW_PARSE_PLPGSQL_EXPR); + if (list_length(parseResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot re-parse constraint expr into a raw expression"))); + + if (IsA(linitial(parseResult), RawStmt)) + { + RawStmt *stmt = linitial(parseResult); + + if (IsA(stmt->stmt, SelectStmt)) + { + SelectStmt *select = (SelectStmt *) stmt->stmt; + + if (list_length(select->targetList) == 1 && + IsA(linitial(select->targetList), ResTarget)) + { + ResTarget *resTarget = linitial(select->targetList); + + return resTarget->val; + } + } + } + + ereport(ERROR, + errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("re-parsed expr does not match the expected structure")); +} + +/* + * ALTER COLUMN col ADD GENERATED USING CONSTRAINT name STORED + * + * Change a regular column into a stored generated column without a table + * rewrite, using the expression contained in the given constraint. + * + * The constraint must be a CHECK constraint proving that the expression is + * already satisfied by all the values in the column (see + * findUsableConstraintForAddGenStored). + */ +static ObjectAddress +ATExecAddGeneratedStored(AlteredTableInfo *tab, + Relation rel, + const char *colName, + Constraint *def) +{ + HeapTuple tuple, + newtuple; + Form_pg_attribute attTup; + AttrNumber attnum; + Bitmapset *colRefs; + bool is_expr; + ObjectAddress address; + Relation pg_attribute; + Node *foundConstraintExpr = NULL; + AddGenConstrError reason; + Node *newRawDefExpr; + RawColumnDefault *rawDefault; + List *cookedResult = NIL; + Datum repl_val[Natts_pg_attribute]; + bool repl_null[Natts_pg_attribute]; + bool repl_repl[Natts_pg_attribute]; + + Assert(def->raw_expr == NULL); + Assert(def->cooked_expr == NULL); + Assert(def->conname != NULL); + Assert(def->generated_when == ATTRIBUTE_IDENTITY_ALWAYS); + Assert(def->generated_kind == ATTRIBUTE_GENERATED_STORED); + + tuple = SearchSysCacheAttName(RelationGetRelid(rel), colName); + if (!HeapTupleIsValid(tuple)) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_COLUMN), + errmsg("column \"%s\" of relation \"%s\" does not exist", + colName, RelationGetRelationName(rel)))); + + attTup = (Form_pg_attribute) GETSTRUCT(tuple); + + attnum = attTup->attnum; + if (attnum <= 0) + ereport(ERROR, + (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("cannot alter system column \"%s\"", + colName))); + + if (attTup->attidentity) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("Column \"%s\" of relation \"%s\" is an identity column.", + colName, RelationGetRelationName(rel)))); + + if (attTup->attgenerated) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("Column \"%s\" of relation \"%s\" is already a generated column.", + colName, RelationGetRelationName(rel)))); + + /* + * This column might be referenced directly in a partition key, or through + * a whole-row expression. + */ + colRefs = bms_make_singleton(attnum - FirstLowInvalidHeapAttributeNumber); + colRefs = bms_add_member(colRefs, 0 - FirstLowInvalidHeapAttributeNumber); + if (has_partition_attrs(rel, colRefs, &is_expr)) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("Column \"%s\" is referenced in the partition key of relation \"%s\".", + colName, RelationGetRelationName(rel)))); + + checkDependenciesForAddGenStored(rel, attnum, colName); + + /* + * Now, try to find the constraint by name, and see if it has the + * necessary structure to prove that the values are consistent. + */ + foundConstraintExpr = findUsableConstraintForAddGenStored(rel, attnum, + def->conname, &reason); + if (foundConstraintExpr == NULL) + { + if (reason == ADD_GEN_CONSTR_NOT_VALID) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("The constraint \"%s\" is not valid.", def->conname)); + if (reason == ADD_GEN_CONSTR_SHAPE_MISMATCH) + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("The constraint \"%s\" is not of the form CHECK ( pg_datum_image_equal(\"%s\", expr) ).", + def->conname, colName)); + ereport(ERROR, + errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("cannot convert column \"%s\" to generated", colName), + errdetail("Could not find CHECK constraint \"%s\".", def->conname)); + } + + /** + * Update pg_attribute: mark the column as generated stored, and clear + * atthasmissing and attmissingval (they can never be set for a column which + * has been originally defined as generated). */ + pg_attribute = table_open(AttributeRelationId, RowExclusiveLock); + + memset(repl_val, 0, sizeof(repl_val)); + memset(repl_null, false, sizeof(repl_null)); + memset(repl_repl, false, sizeof(repl_repl)); + + repl_val[Anum_pg_attribute_attgenerated - 1] = CharGetDatum(ATTRIBUTE_GENERATED_STORED); + repl_val[Anum_pg_attribute_atthasmissing - 1] = BoolGetDatum(false); + repl_null[Anum_pg_attribute_attmissingval - 1] = true; + + repl_repl[Anum_pg_attribute_attgenerated - 1] = true; + repl_repl[Anum_pg_attribute_atthasmissing - 1] = true; + repl_repl[Anum_pg_attribute_attmissingval - 1] = true; + + newtuple = heap_modify_tuple(tuple, RelationGetDescr(pg_attribute), + repl_val, repl_null, repl_repl); + + CatalogTupleUpdate(pg_attribute, &tuple->t_self, newtuple); + + table_close(pg_attribute, RowExclusiveLock); + + heap_freetuple(newtuple); + ReleaseSysCache(tuple); + + /* Make above changes visible */ + CommandCounterIncrement(); + + /* Recover a raw parse tree for the expression found in the constraint */ + newRawDefExpr = reconstructRawExpr(rel, foundConstraintExpr); + + /* + * Remove previous default value, if any, and store the new generator + * expression. + */ + RemoveAttrDefault(RelationGetRelid(rel), attnum, DROP_RESTRICT, + false, false); + + rawDefault = palloc0_object(RawColumnDefault); + rawDefault->attnum = attnum; + rawDefault->raw_default = newRawDefExpr; + rawDefault->generated = ATTRIBUTE_GENERATED_STORED; + + cookedResult = AddRelationNewConstraints(rel, list_make1(rawDefault), NIL, + false /* allow_merge */ , + true /* is_local */ , + false /* is_internal */ , + NULL /* queryString */ ); + + if (list_length(cookedResult) != 1) + ereport(ERROR, + (errcode(ERRCODE_INTERNAL_ERROR), + errmsg_internal("cannot store constraint as default value"))); + + InvokeObjectPostAlterHook(RelationRelationId, + RelationGetRelid(rel), attnum); + + ObjectAddressSubSet(address, RelationRelationId, + RelationGetRelid(rel), attnum); + return address; +} + /* * ALTER TABLE ALTER COLUMN DROP EXPRESSION */ diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y index 0563453fe24..55a8807c2de 100644 --- a/src/backend/parser/gram.y +++ b/src/backend/parser/gram.y @@ -2642,6 +2642,24 @@ alter_table_cmd: n->name = $3; n->def = (Node *) c; + $$ = (Node *) n; + } + /* ALTER TABLE <name> ALTER [COLUMN] <colname> ADD GENERATED USING CONSTRAINT constraint_name STORED */ + | ALTER opt_column ColId ADD_P GENERATED USING CONSTRAINT name STORED + { + AlterTableCmd *n = makeNode(AlterTableCmd); + Constraint *c = makeNode(Constraint); + + c->conname = $8; + c->contype = CONSTR_GENERATED; + c->generated_when = ATTRIBUTE_IDENTITY_ALWAYS; + c->generated_kind = ATTRIBUTE_GENERATED_STORED; + c->location = @8; + + n->subtype = AT_AddGeneratedStored; + n->name = $3; + n->def = (Node *) c; + $$ = (Node *) n; } /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET <sequence options>/RESET */ diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c index 93a67e53026..e867cf329a3 100644 --- a/src/backend/utils/adt/pseudotypes.c +++ b/src/backend/utils/adt/pseudotypes.c @@ -398,12 +398,12 @@ pg_datum_image_equal(PG_FUNCTION_ARGS) } else { - Oid typ; - Datum arg0; - Datum arg1; - bool typbyval; - char typalign; - int16 typlen; + Oid typ; + Datum arg0; + Datum arg1; + bool typbyval; + char typalign; + int16 typlen; typ = get_fn_expr_argtype(fcinfo->flinfo, 0); diff --git a/src/bin/psql/t/010_tab_completion.pl b/src/bin/psql/t/010_tab_completion.pl index 1d2e5f5b92a..32871f14aad 100644 --- a/src/bin/psql/t/010_tab_completion.pl +++ b/src/bin/psql/t/010_tab_completion.pl @@ -44,7 +44,10 @@ $node->safe_psql('postgres', . "CREATE TABLE mytab246 (f1 int, f2 text);\n" . "CREATE TABLE \"mixedName\" (f1 int, f2 text);\n" . "CREATE TYPE enum1 AS ENUM ('foo', 'bar', 'baz', 'BLACK');\n" - . "CREATE PUBLICATION some_publication;\n"); + . "CREATE PUBLICATION some_publication;\n" + . "CREATE TABLE gencol_test (a int primary key, b int);\n" + . "ALTER TABLE gencol_test ADD CONSTRAINT check_gen CHECK (pg_datum_image_equal(b, a + 1));\n" +); # In a VPATH build, we'll be started in the source directory, but we want # to run in the build directory so that we can use relative paths to @@ -422,6 +425,23 @@ check_completion( clear_line(); +check_completion("ALTER TABLE gencol_test ALTER COLUMN b A\t", qr/ADD /, + "complete ALTER COLUMN <col> ADD"); + +check_completion("G\t", qr/GENERATED /, + "complete ALTER COLUMN <col> ADD GENERATED"); + +check_completion("U\t", qr/USING CONSTRAINT /, + "complete ALTER COLUMN <col> ADD GENERATED USING CONSTRAINT"); + +check_completion("\t", qr/check_gen /, + "complete ALTER COLUMN <col> ADD GENERATED USING CONSTRAINT offers check constraint names"); + +check_completion("S\t", qr/STORED /, + "complete ALTER COLUMN <col> ADD GENERATED USING CONSTRAINT constr_name STORED"); + +clear_query(); + # send psql an explicit \q to shut it down, else pty won't close properly $h->quit or die "psql returned $?"; diff --git a/src/bin/psql/tab-complete.in.c b/src/bin/psql/tab-complete.in.c index 96de78cd087..00fc9d20950 100644 --- a/src/bin/psql/tab-complete.in.c +++ b/src/bin/psql/tab-complete.in.c @@ -489,6 +489,15 @@ static const SchemaQuery Query_for_constraint_of_table_not_validated = { .refnamespace = "c1.relnamespace", }; +static const SchemaQuery Query_for_check_constraint_of_table = { + .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_class c1", + .selcondition = "con.conrelid=c1.oid and con.contype='c'", + .result = "con.conname", + .refname = "c1.relname", + .refviscondition = "pg_catalog.pg_table_is_visible(c1.oid)", + .refnamespace = "c1.relnamespace", +}; + static const SchemaQuery Query_for_constraint_of_type = { .catname = "pg_catalog.pg_constraint con, pg_catalog.pg_type t", .selcondition = "con.contypid=t.oid", @@ -2951,13 +2960,39 @@ match_previous_words(int pattern_id, /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED")) - COMPLETE_WITH("ALWAYS", "BY DEFAULT"); - /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED */ + COMPLETE_WITH("ALWAYS", "BY DEFAULT", "USING CONSTRAINT"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED ALWAYS */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS") || - Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || + Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "ALWAYS")) + COMPLETE_WITH("AS IDENTITY"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED BY DEFAULT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "BY", "DEFAULT")) COMPLETE_WITH("AS IDENTITY"); + /* ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED USING CONSTRAINT */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev8_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED USING CONSTRAINT + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "USING", "CONSTRAINT")) + { + set_completion_reference(prev7_wd); + COMPLETE_WITH_SCHEMA_QUERY(Query_for_check_constraint_of_table); + } + + /* + * ALTER TABLE ALTER [COLUMN] <foo> ADD GENERATED USING CONSTRAINT + * constr_name + */ + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "ADD", "GENERATED", "USING", "CONSTRAINT", MatchAny)) + COMPLETE_WITH("STORED"); + else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "ADD", "GENERATED", "USING", "CONSTRAINT", MatchAny)) + COMPLETE_WITH("STORED"); /* ALTER TABLE ALTER [COLUMN] <foo> SET */ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") || Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET")) diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h index 0debcd193ab..7c9a89d1f9d 100644 --- a/src/include/nodes/parsenodes.h +++ b/src/include/nodes/parsenodes.h @@ -2463,6 +2463,7 @@ typedef enum AlterTableType AT_CookedColumnDefault, /* add a pre-cooked column default */ AT_DropNotNull, /* alter column drop not null */ AT_SetNotNull, /* alter column set not null */ + AT_AddGeneratedStored, /* add generated using constraint */ AT_SetExpression, /* alter column set expression */ AT_DropExpression, /* alter column drop expression */ AT_SetStatistics, /* alter column set statistics */ diff --git a/src/test/modules/injection_points/Makefile b/src/test/modules/injection_points/Makefile index 8e2dfaa2384..7e983eb5d01 100644 --- a/src/test/modules/injection_points/Makefile +++ b/src/test/modules/injection_points/Makefile @@ -11,7 +11,7 @@ PGFILEDESC = "injection_points - facility for injection points" EXTRA_INSTALL = contrib/test_decoding -REGRESS = injection_points hashagg reindex_conc vacuum +REGRESS = injection_points hashagg reindex_conc vacuum alter_table REGRESS_OPTS = --dlpath=$(top_builddir)/src/test/regress # wait_cleanup terminates another backend, whose FATAL message can be lost diff --git a/src/test/modules/injection_points/expected/alter_table.out b/src/test/modules/injection_points/expected/alter_table.out new file mode 100644 index 00000000000..5f53b2a29c3 --- /dev/null +++ b/src/test/modules/injection_points/expected/alter_table.out @@ -0,0 +1,36 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; +SELECT injection_points_set_local(); + injection_points_set_local +---------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + injection_points_attach +------------------------- + +(1 row) + +CREATE SCHEMA testgen_inj; +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (pg_datum_image_equal(b, a * 2)); +NOTICE: notice triggered for injection point alter-table-phase-3-verify +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED USING CONSTRAINT c2 STORED; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; +DROP EXTENSION injection_points; diff --git a/src/test/modules/injection_points/meson.build b/src/test/modules/injection_points/meson.build index 97e7a736282..e6058fd04b0 100644 --- a/src/test/modules/injection_points/meson.build +++ b/src/test/modules/injection_points/meson.build @@ -64,6 +64,7 @@ tests += { 'hashagg', 'reindex_conc', 'vacuum', + 'alter_table', ], 'regress_args': ['--dlpath', meson.project_build_root() / 'src/test/regress'], # The injection points are cluster-wide, so disable installcheck diff --git a/src/test/modules/injection_points/sql/alter_table.sql b/src/test/modules/injection_points/sql/alter_table.sql new file mode 100644 index 00000000000..135204e1f7a --- /dev/null +++ b/src/test/modules/injection_points/sql/alter_table.sql @@ -0,0 +1,24 @@ +-- Tests for ALTER TABLE +CREATE EXTENSION injection_points; + +SELECT injection_points_set_local(); + +SELECT injection_points_attach('alter-table-phase-3-rewrite', 'notice'); +SELECT injection_points_attach('alter-table-phase-3-verify', 'notice'); + +CREATE SCHEMA testgen_inj; + +-- Check that the table isn't being rewritten nor scanned during phase 3, +-- even if other objects depend on the column we are changing. +CREATE TABLE testgen_inj.t1 (a INT, b INT NOT NULL); +INSERT INTO testgen_inj.t1 (a, b) VALUES (1, 2); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c1 CHECK (b > 0); +ALTER TABLE testgen_inj.t1 ADD CONSTRAINT c2 CHECK (pg_datum_image_equal(b, a * 2)); +CREATE INDEX ON testgen_inj.t1 (b); +ALTER TABLE testgen_inj.t1 ALTER b + ADD GENERATED USING CONSTRAINT c2 STORED; +-- we expect to *not* see a "alter-table-phase-3-*" notice here +DROP TABLE testgen_inj.t1; +DROP SCHEMA testgen_inj; + +DROP EXTENSION injection_points; diff --git a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c index 105872a3517..47b8337307d 100644 --- a/src/test/modules/test_ddl_deparse/test_ddl_deparse.c +++ b/src/test/modules/test_ddl_deparse/test_ddl_deparse.c @@ -129,6 +129,9 @@ get_altertable_subcmdinfo(PG_FUNCTION_ARGS) case AT_SetNotNull: strtype = "SET NOT NULL"; break; + case AT_AddGeneratedStored: + strtype = "ADD GENERATED STORED"; + break; case AT_SetExpression: strtype = "SET EXPRESSION"; break; diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index 20707c186b3..f69ab467c71 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -4901,3 +4901,461 @@ drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; NOTICE: drop cascades to table alter2.t1 +-- Tests for ALTER COLUMN ... ADD GENERATED USING CONSTRAINT name STORED +-- turning a regular column into a stored generated column without a rewrite +create schema tgen; +create table tgen.t1 (a int, b int); +insert into tgen.t1 (a, b) + select x, x * 2 from generate_series(1, 5) x; +begin; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +\d tgen.t1 + Table "tgen.t1" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+------------------------------------ + a | integer | | | + b | integer | | | generated always as (a * 2) stored +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(b, a * 2)) + +insert into tgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +rollback; +-- test that we accept the constraint with the operands swapped, too +begin; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(a * 2, b)); +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +\d tgen.t1 + Table "tgen.t1" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+------------------------------------ + a | integer | | | + b | integer | | | generated always as (a * 2) stored +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(a * 2, b)) + +insert into tgen.t1 (a, b) values (10, 20); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +rollback; +begin; +update tgen.t1 set b = a; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(a, b)); +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +\d tgen.t1 + Table "tgen.t1" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+-------------------------------- + a | integer | | | + b | integer | | | generated always as (a) stored +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(a, b)) + +insert into tgen.t1 (a, b) values (10, 10); +ERROR: cannot insert a non-DEFAULT value into column "b" +DETAIL: Column "b" is a generated column. +rollback; +-- check that neither the table nor indexes are rewritten +begin; +alter table tgen.t1 + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +create index t1_a_idx on tgen.t1 (a); +select pg_relation_filenode('tgen.t1') as t1_filenode_before \gset +select pg_relation_filenode('tgen.t1_a_idx') as t1_idx_filenode_before \gset +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +select pg_relation_filenode('tgen.t1') as t1_filenode_after \gset +select pg_relation_filenode('tgen.t1_a_idx') as t1_idx_filenode_after \gset +select :t1_filenode_before = :t1_filenode_after as did_skip_table_rewrite, + :t1_idx_filenode_before = :t1_idx_filenode_after as did_skip_idx_rewrite; + did_skip_table_rewrite | did_skip_idx_rewrite +------------------------+---------------------- + t | t +(1 row) + +rollback; +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks. The t_migration database is intentionally +-- left behind for pg_dump/pg_restore tests. +create table tgen.t_migration (a int); +select pg_relation_filenode('tgen.t_migration') as t_migration_filenode_before \gset +insert into tgen.t_migration select x from generate_series(1, 5) x; +-- test nulls, too +insert into tgen.t_migration (a) values (null); +alter table tgen.t_migration add column b int; +-- take care of new and updated columns +create function tgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger tgen_trig + before insert or update on tgen.t_migration + for each row execute function tgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table tgen.t_migration + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)) not valid; +select locktype, mode from pg_locks + where relation = 'tgen.t_migration'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +insert into tgen.t_migration (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update tgen.t_migration set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table tgen.t_migration validate constraint chk_gen; +select locktype, mode from pg_locks + where relation = 'tgen.t_migration'::regclass and granted; + locktype | mode +----------+-------------------------- + relation | ShareUpdateExclusiveLock +(1 row) + +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table tgen.t_migration alter column b + add generated using constraint chk_gen stored; +select locktype, mode from pg_locks +where relation = 'tgen.t_migration'::regclass and granted; + locktype | mode +----------+--------------------- + relation | AccessExclusiveLock +(1 row) + +commit; +select pg_relation_filenode('tgen.t_migration') as t_migration_filenode_after \gset +select :t_migration_filenode_before = :t_migration_filenode_after as did_skip_rewrite; + did_skip_rewrite +------------------ + t +(1 row) + +select * from tgen.t_migration; + a | b +-----+----- + 100 | 200 + 200 | 400 + 300 | 600 + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + | +(9 rows) + +-- verify that it's still possible to insert rows (the trigger is still +-- installed at this point) +insert into tgen.t_migration (a) values (400); +insert into tgen.t_migration (a) values (500); +\d tgen.t_migration + Table "tgen.t_migration" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+------------------------------------ + a | integer | | | + b | integer | | | generated always as (a * 2) stored +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(b, a * 2)) +Triggers: + tgen_trig BEFORE INSERT OR UPDATE ON tgen.t_migration FOR EACH ROW EXECUTE FUNCTION tgen.gen() + +select * from tgen.t_migration order by a nulls first; + a | b +-----+------ + | + 1 | 2 + 2 | 4 + 3 | 6 + 4 | 8 + 5 | 10 + 100 | 200 + 200 | 400 + 300 | 600 + 400 | 800 + 500 | 1000 +(11 rows) + +-- test support for partitioned tables +create table tgen.tpart (a int, b int) partition by hash (a); +alter table tgen.tpart + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +create table tgen.tpart_p1 partition of tgen.tpart + for values with (modulus 2, remainder 0); +create table tgen.tpart_p2 partition of tgen.tpart + for values with (modulus 2, remainder 1); +insert into tgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; +-- altering a single partition is not allowed +alter table tgen.tpart_p1 alter column b + add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Converting only part of a partitioning or inheritance hierarchy is not supported. +HINT: Use this command on the root table without specifying ONLY. +-- altering only the parent table is not allowed +alter table only tgen.tpart alter column b + add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Converting only part of a partitioning or inheritance hierarchy is not supported. +HINT: Use this command on the root table without specifying ONLY. +-- altering the parent table, recursing +alter table tgen.tpart alter column b + add generated using constraint chk_gen stored; +-- expected: all the partitions have been changed +\d tgen.tpart + Partitioned table "tgen.tpart" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+------------------------------------ + a | integer | | | + b | integer | | | generated always as (a * 2) stored +Partition key: HASH (a) +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(b, a * 2)) +Number of partitions: 2 (Use \d+ to list them.) + +\d tgen.tpart_p1 + Table "tgen.tpart_p1" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+------------------------------------ + a | integer | | | + b | integer | | | generated always as (a * 2) stored +Partition of: tgen.tpart FOR VALUES WITH (modulus 2, remainder 0) +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(b, a * 2)) + +\d tgen.tpart_p2 + Table "tgen.tpart_p2" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+------------------------------------ + a | integer | | | + b | integer | | | generated always as (a * 2) stored +Partition of: tgen.tpart FOR VALUES WITH (modulus 2, remainder 1) +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(b, a * 2)) + +-- tpart intentionally left behind for pg_dump/pg_restore tests +-- test support for inheritance and subpartitions +create table tgen.root (a int, b int, c int, + constraint chk_gen check ( pg_datum_image_equal(c, a+b) )); +create table tgen.intermediate () inherits (tgen.root); +create table tgen.leaf () inherits (tgen.intermediate); +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table tgen.root alter column c + add generated using constraint chk_gen stored; +rollback; +-- ... hence all these should result in an error +alter table only tgen.root alter column c + add generated using constraint chk_gen stored; +ERROR: cannot convert column "c" to generated +DETAIL: Converting only part of a partitioning or inheritance hierarchy is not supported. +HINT: Use this command on the root table without specifying ONLY. +alter table tgen.intermediate alter column c + add generated using constraint chk_gen stored; +ERROR: cannot convert column "c" to generated +DETAIL: Converting only part of a partitioning or inheritance hierarchy is not supported. +HINT: Use this command on the root table without specifying ONLY. +alter table only tgen.intermediate alter column c + add generated using constraint chk_gen stored; +ERROR: cannot convert column "c" to generated +DETAIL: Converting only part of a partitioning or inheritance hierarchy is not supported. +HINT: Use this command on the root table without specifying ONLY. +alter table tgen.leaf alter column c + add generated using constraint chk_gen stored; +ERROR: cannot convert column "c" to generated +DETAIL: Converting only part of a partitioning or inheritance hierarchy is not supported. +HINT: Use this command on the root table without specifying ONLY. +alter table only tgen.leaf alter column c + add generated using constraint chk_gen stored; +ERROR: cannot convert column "c" to generated +DETAIL: Converting only part of a partitioning or inheritance hierarchy is not supported. +HINT: Use this command on the root table without specifying ONLY. +drop table tgen.root cascade; +NOTICE: drop cascades to 2 other objects +DETAIL: drop cascades to table tgen.intermediate +drop cascades to table tgen.leaf +-- tests for invalid invocations +-- invalid: only supports STORED. The following are syntax errors. +alter table tgen.t1 alter column b add generated using constraint chk_gen; +ERROR: syntax error at or near ";" +LINE 1: ...en.t1 alter column b add generated using constraint chk_gen; + ^ +alter table tgen.t1 alter column b add generated using constraint chk_gen virtual; +ERROR: syntax error at or near "virtual" +LINE 1: ...ter column b add generated using constraint chk_gen virtual; + ^ +-- invalid: b is already a generated column +begin; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +alter table tgen.t1 alter b add generated using constraint chk_gen stored; +alter table tgen.t1 alter b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" of relation "t1" is already a generated column. +rollback; +-- invalid: b is already a generated column +create table tgen.t2 (a int, b int not null generated always as (a * 2) stored); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" of relation "t2" is already a generated column. +drop table tgen.t2; +-- invalid: b is an identity column +create table tgen.t2 (a int, b int generated always as identity); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, 1)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" of relation "t2" is an identity column. +drop table tgen.t2; +create table tgen.t2 (a int, b int generated by default as identity); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, 1)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" of relation "t2" is an identity column. +drop table tgen.t2; +-- invalid: b is a serial column +create table tgen.t2 (a bigint, b bigserial); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, 1::bigint)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" of relation "t2" is a serial column (depends on "sequence tgen.t2_b_seq"). +drop table tgen.t2; +-- invalid: c is referenced by another generated column +create table tgen.t2 (a int, b int generated always as (c + 1), c int); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(c, 1)); +alter table tgen.t2 alter column c add generated using constraint chk_gen stored; +ERROR: cannot convert column "c" to generated +DETAIL: Column "c" is referenced by generated column "b". +drop table tgen.t2; +-- invalid: c references another generated column +create table tgen.t2 (a int, b int generated always as (a + 1), c int); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(c, b + 1)); +alter table tgen.t2 alter column c add generated using constraint chk_gen stored; +ERROR: cannot use generated column "b" in column generation expression +DETAIL: A generated column cannot reference another generated column. +drop table tgen.t2; +-- invalid: b is referenced in a partition key +create table tgen.t2 (a int, b int not null) partition by hash (b); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t2 alter b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" is referenced in the partition key of relation "t2". +drop table tgen.t2; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +-- the two tables are intentionally left behind for pg_dump/pg_restore tests +show search_path \gset +set search_path to tgen, public; +create table t_wholerow_1 (a int, b int) partition by range ((t_wholerow_1)); +alter table tgen.t_wholerow_1 add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t_wholerow_1 alter b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" is referenced in the partition key of relation "t_wholerow_1". +create table t_wholerow_2 (a int, b int) partition by range ((t_wholerow_2 is null)); +alter table tgen.t_wholerow_2 add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t_wholerow_2 alter b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" is referenced in the partition key of relation "t_wholerow_2". +set search_path to :search_path; +-- fails when the constraint does not exist +alter table tgen.t1 alter column b + add generated using constraint chk_gen_does_not_exist stored; +ERROR: cannot convert column "b" to generated +DETAIL: Could not find CHECK constraint "chk_gen_does_not_exist". +-- fails when the constraint is not valid +begin; +alter table tgen.t1 + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)) not valid; +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: The constraint "chk_gen" is not valid. +rollback; +-- fails when the constraint is not enforced +begin; +alter table tgen.t1 + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)) not enforced; +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: The constraint "chk_gen" is not valid. +rollback; +-- fails when the constraint exists but doesn't have the expected shape +begin; +alter table tgen.t1 add constraint chk_gen check (b = a * 2); +alter table tgen.t1 alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: The constraint "chk_gen" is not of the form CHECK ( pg_datum_image_equal("b", expr) ). +rollback; +begin; +alter table tgen.t1 add constraint chk_gen check (b >= a * 2); +alter table tgen.t1 alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: The constraint "chk_gen" is not of the form CHECK ( pg_datum_image_equal("b", expr) ). +rollback; +create table tgen.t2 (a int, b int); +-- invalid: expr must be immutable +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, (a + random())::int)); +alter table tgen.t2 alter column b + add generated using constraint chk_gen stored; +ERROR: generation expression is not immutable +drop table tgen.t2; +drop table tgen.t1; +-- invalid: the column is part of a foreign key with on delete +-- or on update actions +create table tgen.tpkey (a int primary key); +create table tgen.tfkey (a int primary key, b int references tgen.tpkey(a) on delete set null); +alter table tgen.tfkey add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.tfkey alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" is part of a foreign key with an ON DELETE / ON UPDATE action. +drop table tgen.tfkey; +create table tgen.tfkey (a int primary key, b int references tgen.tpkey(a) on update cascade); +alter table tgen.tfkey add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.tfkey alter column b add generated using constraint chk_gen stored; +ERROR: cannot convert column "b" to generated +DETAIL: Column "b" is part of a foreign key with an ON DELETE / ON UPDATE action. +drop table tgen.tfkey; +drop table tgen.tpkey; +-- intentionally left behind for pg_dump/pg_restore tests +create table tgen.t_simple (a int, b int not null); +insert into tgen.t_simple (a, b) values (1, 2), (2, 3); +alter table tgen.t_simple add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t_simple alter b add generated using constraint chk_gen stored; +\d tgen.t_simple + Table "tgen.t_simple" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+------------------------------------ + a | integer | | | + b | integer | | not null | generated always as (a + 1) stored +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(b, a + 1)) + +-- intentionally left behind for pg_dump/pg_restore tests +create table tgen.t_tsvect (a text, b text, c tsvector); +insert into tgen.t_tsvect (a, b) values ('foo', 'bar'), ('bar', 'baz'); +update tgen.t_tsvect set c = to_tsvector('english', a || ' ' || b); +alter table tgen.t_tsvect add constraint chk_gen + check ( pg_datum_image_equal(c, to_tsvector('english', a || ' ' || b)) ); +alter table tgen.t_tsvect alter c add generated using constraint chk_gen stored; +\d tgen.t_tsvect + Table "tgen.t_tsvect" + Column | Type | Collation | Nullable | Default +--------+----------+-----------+----------+--------------------------------------------------------------------------------------- + a | text | | | + b | text | | | + c | tsvector | | | generated always as (to_tsvector('english'::regconfig, (a || ' '::text) || b)) stored +Check constraints: + "chk_gen" CHECK (pg_datum_image_equal(c, to_tsvector('english'::regconfig, (a || ' '::text) || b))) + diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 18f3fdac57a..871c90e3925 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -3187,3 +3187,293 @@ alter table alter1.t1 set schema alter2; drop publication pub1; drop schema alter1 cascade; drop schema alter2 cascade; + +-- Tests for ALTER COLUMN ... ADD GENERATED USING CONSTRAINT name STORED +-- turning a regular column into a stored generated column without a rewrite +create schema tgen; + +create table tgen.t1 (a int, b int); +insert into tgen.t1 (a, b) + select x, x * 2 from generate_series(1, 5) x; + +begin; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +\d tgen.t1 +insert into tgen.t1 (a, b) values (10, 20); +rollback; + +-- test that we accept the constraint with the operands swapped, too +begin; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(a * 2, b)); +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +\d tgen.t1 +insert into tgen.t1 (a, b) values (10, 20); +rollback; +begin; +update tgen.t1 set b = a; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(a, b)); +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +\d tgen.t1 +insert into tgen.t1 (a, b) values (10, 10); +rollback; + +-- check that neither the table nor indexes are rewritten +begin; +alter table tgen.t1 + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +create index t1_a_idx on tgen.t1 (a); +select pg_relation_filenode('tgen.t1') as t1_filenode_before \gset +select pg_relation_filenode('tgen.t1_a_idx') as t1_idx_filenode_before \gset +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +select pg_relation_filenode('tgen.t1') as t1_filenode_after \gset +select pg_relation_filenode('tgen.t1_a_idx') as t1_idx_filenode_after \gset +select :t1_filenode_before = :t1_filenode_after as did_skip_table_rewrite, + :t1_idx_filenode_before = :t1_idx_filenode_after as did_skip_idx_rewrite; +rollback; + +-- test the whole process for adding a stored generated column without +-- long-lived exclusive locks. The t_migration database is intentionally +-- left behind for pg_dump/pg_restore tests. +create table tgen.t_migration (a int); +select pg_relation_filenode('tgen.t_migration') as t_migration_filenode_before \gset +insert into tgen.t_migration select x from generate_series(1, 5) x; +-- test nulls, too +insert into tgen.t_migration (a) values (null); +alter table tgen.t_migration add column b int; +-- take care of new and updated columns +create function tgen.gen () returns trigger language plpgsql as $$ +begin + new.b = new.a * 2; return new; +end +$$; +create trigger tgen_trig + before insert or update on tgen.t_migration + for each row execute function tgen.gen(); +-- add the constraint as not valid: enforced only for new and updated rows +begin; +alter table tgen.t_migration + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)) not valid; +select locktype, mode from pg_locks + where relation = 'tgen.t_migration'::regclass and granted; +commit; +insert into tgen.t_migration (a) values (100), (200), (300); +-- backfill existing rows at the appropriate pace +update tgen.t_migration set b = a * 2 where b is null; +-- validate: this scans the table, but without an exclusive lock +begin; +alter table tgen.t_migration validate constraint chk_gen; +select locktype, mode from pg_locks + where relation = 'tgen.t_migration'::regclass and granted; +commit; +-- now the schema update, which doesn't need to rewrite the table thanks to +-- the constraint +begin; +alter table tgen.t_migration alter column b + add generated using constraint chk_gen stored; +select locktype, mode from pg_locks +where relation = 'tgen.t_migration'::regclass and granted; +commit; +select pg_relation_filenode('tgen.t_migration') as t_migration_filenode_after \gset +select :t_migration_filenode_before = :t_migration_filenode_after as did_skip_rewrite; +select * from tgen.t_migration; +-- verify that it's still possible to insert rows (the trigger is still +-- installed at this point) +insert into tgen.t_migration (a) values (400); +insert into tgen.t_migration (a) values (500); +\d tgen.t_migration +select * from tgen.t_migration order by a nulls first; + +-- test support for partitioned tables +create table tgen.tpart (a int, b int) partition by hash (a); +alter table tgen.tpart + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +create table tgen.tpart_p1 partition of tgen.tpart + for values with (modulus 2, remainder 0); +create table tgen.tpart_p2 partition of tgen.tpart + for values with (modulus 2, remainder 1); +insert into tgen.tpart (a, b) select x, x * 2 from generate_series(1, 5) x; + +-- altering a single partition is not allowed +alter table tgen.tpart_p1 alter column b + add generated using constraint chk_gen stored; + +-- altering only the parent table is not allowed +alter table only tgen.tpart alter column b + add generated using constraint chk_gen stored; + +-- altering the parent table, recursing +alter table tgen.tpart alter column b + add generated using constraint chk_gen stored; +-- expected: all the partitions have been changed +\d tgen.tpart +\d tgen.tpart_p1 +\d tgen.tpart_p2 + +-- tpart intentionally left behind for pg_dump/pg_restore tests + +-- test support for inheritance and subpartitions +create table tgen.root (a int, b int, c int, + constraint chk_gen check ( pg_datum_image_equal(c, a+b) )); +create table tgen.intermediate () inherits (tgen.root); +create table tgen.leaf () inherits (tgen.intermediate); + +-- it's only allowed to change the whole hierarchy at once... +begin; +alter table tgen.root alter column c + add generated using constraint chk_gen stored; +rollback; + +-- ... hence all these should result in an error +alter table only tgen.root alter column c + add generated using constraint chk_gen stored; +alter table tgen.intermediate alter column c + add generated using constraint chk_gen stored; +alter table only tgen.intermediate alter column c + add generated using constraint chk_gen stored; +alter table tgen.leaf alter column c + add generated using constraint chk_gen stored; +alter table only tgen.leaf alter column c + add generated using constraint chk_gen stored; + +drop table tgen.root cascade; + +-- tests for invalid invocations +-- invalid: only supports STORED. The following are syntax errors. +alter table tgen.t1 alter column b add generated using constraint chk_gen; +alter table tgen.t1 alter column b add generated using constraint chk_gen virtual; + +-- invalid: b is already a generated column +begin; +alter table tgen.t1 add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +alter table tgen.t1 alter b add generated using constraint chk_gen stored; +alter table tgen.t1 alter b add generated using constraint chk_gen stored; +rollback; + +-- invalid: b is already a generated column +create table tgen.t2 (a int, b int not null generated always as (a * 2) stored); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, a * 2)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +drop table tgen.t2; + +-- invalid: b is an identity column +create table tgen.t2 (a int, b int generated always as identity); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, 1)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +drop table tgen.t2; +create table tgen.t2 (a int, b int generated by default as identity); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, 1)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +drop table tgen.t2; + +-- invalid: b is a serial column +create table tgen.t2 (a bigint, b bigserial); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, 1::bigint)); +alter table tgen.t2 alter column b add generated using constraint chk_gen stored; +drop table tgen.t2; + +-- invalid: c is referenced by another generated column +create table tgen.t2 (a int, b int generated always as (c + 1), c int); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(c, 1)); +alter table tgen.t2 alter column c add generated using constraint chk_gen stored; +drop table tgen.t2; + +-- invalid: c references another generated column +create table tgen.t2 (a int, b int generated always as (a + 1), c int); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(c, b + 1)); +alter table tgen.t2 alter column c add generated using constraint chk_gen stored; +drop table tgen.t2; + +-- invalid: b is referenced in a partition key +create table tgen.t2 (a int, b int not null) partition by hash (b); +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t2 alter b add generated using constraint chk_gen stored; +drop table tgen.t2; +-- test for a whole-row reference +-- since it's not possible to reference schema.table in partition by range, +-- temporarily hack the search_path +-- the two tables are intentionally left behind for pg_dump/pg_restore tests +show search_path \gset +set search_path to tgen, public; +create table t_wholerow_1 (a int, b int) partition by range ((t_wholerow_1)); +alter table tgen.t_wholerow_1 add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t_wholerow_1 alter b add generated using constraint chk_gen stored; +create table t_wholerow_2 (a int, b int) partition by range ((t_wholerow_2 is null)); +alter table tgen.t_wholerow_2 add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t_wholerow_2 alter b add generated using constraint chk_gen stored; +set search_path to :search_path; + +-- fails when the constraint does not exist +alter table tgen.t1 alter column b + add generated using constraint chk_gen_does_not_exist stored; + +-- fails when the constraint is not valid +begin; +alter table tgen.t1 + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)) not valid; +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +rollback; + +-- fails when the constraint is not enforced +begin; +alter table tgen.t1 + add constraint chk_gen check (pg_datum_image_equal(b, a * 2)) not enforced; +alter table tgen.t1 alter column b + add generated using constraint chk_gen stored; +rollback; + +-- fails when the constraint exists but doesn't have the expected shape +begin; +alter table tgen.t1 add constraint chk_gen check (b = a * 2); +alter table tgen.t1 alter column b add generated using constraint chk_gen stored; +rollback; +begin; +alter table tgen.t1 add constraint chk_gen check (b >= a * 2); +alter table tgen.t1 alter column b add generated using constraint chk_gen stored; +rollback; + +create table tgen.t2 (a int, b int); +-- invalid: expr must be immutable +alter table tgen.t2 add constraint chk_gen check (pg_datum_image_equal(b, (a + random())::int)); +alter table tgen.t2 alter column b + add generated using constraint chk_gen stored; +drop table tgen.t2; + +drop table tgen.t1; + +-- invalid: the column is part of a foreign key with on delete +-- or on update actions +create table tgen.tpkey (a int primary key); + +create table tgen.tfkey (a int primary key, b int references tgen.tpkey(a) on delete set null); +alter table tgen.tfkey add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.tfkey alter column b add generated using constraint chk_gen stored; +drop table tgen.tfkey; + +create table tgen.tfkey (a int primary key, b int references tgen.tpkey(a) on update cascade); +alter table tgen.tfkey add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.tfkey alter column b add generated using constraint chk_gen stored; +drop table tgen.tfkey; + +drop table tgen.tpkey; + +-- intentionally left behind for pg_dump/pg_restore tests +create table tgen.t_simple (a int, b int not null); +insert into tgen.t_simple (a, b) values (1, 2), (2, 3); +alter table tgen.t_simple add constraint chk_gen check (pg_datum_image_equal(b, a + 1)); +alter table tgen.t_simple alter b add generated using constraint chk_gen stored; +\d tgen.t_simple + +-- intentionally left behind for pg_dump/pg_restore tests +create table tgen.t_tsvect (a text, b text, c tsvector); +insert into tgen.t_tsvect (a, b) values ('foo', 'bar'), ('bar', 'baz'); +update tgen.t_tsvect set c = to_tsvector('english', a || ' ' || b); +alter table tgen.t_tsvect add constraint chk_gen + check ( pg_datum_image_equal(c, to_tsvector('english', a || ' ' || b)) ); +alter table tgen.t_tsvect alter c add generated using constraint chk_gen stored; +\d tgen.t_tsvect diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list index 553162c0810..f6650a8fed1 100644 --- a/src/tools/pgindent/typedefs.list +++ b/src/tools/pgindent/typedefs.list @@ -27,6 +27,7 @@ AcquireSampleRowsFunc ActionList ActiveSnapshotElt AddForeignUpdateTargets_function +AddGenConstrError AddrInfo AffixNode AffixNodeData -- 2.54.0
