On Thu, Jul 31, 2025 at 2:24 AM Corey Huinker <[email protected]> wrote: > >> >> Besides, this does nothing you haven't been able to do for >> decades with expression indexes. > > > What I'm hoping for with this feature is a smoother transition when people > start introducing virtual columns. If there was already an index on that > expression, and some queries start using the new virtual column (with the > same expression), will that cause those queries to miss the index we already > have? If it doesn't, then a customer can roll out the query changes at will > and not need to do some cut-over from using the expression to using the > virtual column. >
hi. I am not sure whether this concern has been addressed, as I am still somewhat confused by the above paragraph. As noted earlier, creating an index on a virtual generated column results in a new index, and that index behaves the same as a regular expression index. An updated and polished patch is attached. The regress tests are quite verbose at the moment, since I make it covered all index types (btree, gist, spgist, hash, gin, and brin). -- jian https://www.enterprisedb.com/
From 0047c05725121fcb8e241a9edb4bd2983fd4dad2 Mon Sep 17 00:00:00 2001 From: jian he <[email protected]> Date: Thu, 8 Jan 2026 14:11:30 +0800 Subject: [PATCH v7 1/1] index on virtual generated column * All index access methods, including btree, hash, GiST, SP-GiST, GIN, and BRIN, are supported. * Primary key and unique indexes on virtual generated columns are not supported. * Exclusion constraints on virtual generated columns are not supported at present and maybe in the future. * Expression indexes and partial (predicate) indexes on virtual generated columns are currently unsupported. * Indexes with INCLUDED columns cannot reference virtual generated columns. * An index on a virtual generated column such as (b INT GENERATED ALWAYS AS (a) VIRTUAL) is effectively equivalent to an index on column a. * Internally, indexes on virtual generated columns are implemented as expression indexes. For example, an index on ``(b INT GENERATED ALWAYS AS (a * 2) VIRTUAL)`` is internally represented as an expression index on ``(a * 2)``. * Additional tests have been added to the pageinspect module to verify index contents for virtual generated columns. * To support ALTER TABLE ... SET EXPRESSION, the pg_index catalog records the the attribute number of the virtual generated column, this is need for indexes rebuilt. ALTER COLUMN SET DATA TYPE triggers table rewrite too; therefore, tracking the attribute number of the virtual generated column referenced by an index is really needed. * if the partitioned table and partition both have an index, then the index over the virtual generated column should be the same expression. For example, the following last command should error out. CREATE TABLE parted (b integer,c integer,a integer GENERATED ALWAYS AS (c+1)) PARTITION BY RANGE (b); CREATE TABLE part (b integer,c integer,a integer GENERATED ALWAYS AS (c)); create index on part(a); create index on parted(a); alter table parted ATTACH partition part for values from (1) to (10); discussion: https://postgr.es/m/CACJufxGao-cypdNhifHAdt8jHfK6-HX=trbovbkgruxw063...@mail.gmail.com discussion: https://postgr.es/m/CACJufxGgkH0PyyqP6ggqcEWHxZzmkV=puY8ad=s8kisss9m...@mail.gmail.com related thread: https://postgr.es/m/[email protected] commitfest: https://commitfest.postgresql.org/patch/5667 --- contrib/pageinspect/expected/btree.out | 11 + contrib/pageinspect/sql/btree.sql | 10 + doc/src/sgml/catalogs.sgml | 15 + src/backend/catalog/index.c | 85 +++++ src/backend/commands/indexcmds.c | 310 +++++++++++++--- src/backend/commands/tablecmds.c | 47 ++- src/backend/executor/execIndexing.c | 12 + src/backend/executor/execPartition.c | 5 + src/backend/parser/parse_utilcmd.c | 29 +- src/backend/utils/adt/ruleutils.c | 34 +- src/include/catalog/index.h | 6 + src/include/catalog/pg_index.h | 3 + src/include/nodes/execnodes.h | 8 + src/test/regress/expected/alter_table.out | 20 ++ .../regress/expected/collate.icu.utf8.out | 13 + src/test/regress/expected/fast_default.out | 8 + .../regress/expected/generated_virtual.out | 334 +++++++++++++++++- src/test/regress/expected/indexing.out | 45 +++ src/test/regress/sql/alter_table.sql | 9 + src/test/regress/sql/collate.icu.utf8.sql | 8 + src/test/regress/sql/fast_default.sql | 6 + src/test/regress/sql/generated_virtual.sql | 146 +++++++- src/test/regress/sql/indexing.sql | 30 ++ 23 files changed, 1114 insertions(+), 80 deletions(-) diff --git a/contrib/pageinspect/expected/btree.out b/contrib/pageinspect/expected/btree.out index 0aa5d73322f..b7d36f7d047 100644 --- a/contrib/pageinspect/expected/btree.out +++ b/contrib/pageinspect/expected/btree.out @@ -183,6 +183,17 @@ tids | SELECT * FROM bt_page_items(get_raw_page('test1_a_idx', 2)); ERROR: block number 2 is out of range for relation "test1_a_idx" +---test index over virtual generated column +CREATE TABLE test4(a int, b int GENERATED ALWAYS AS (a + 1), c text); +INSERT INTO test4(a,c) VALUES (10,11), (10,11); +CREATE INDEX test4_b_idx ON test4 USING btree (b); +CREATE INDEX test4_a_1_idx ON test4 USING btree ((a+1)); +--expect return zero row +SELECT * FROM bt_page_items('test4_b_idx', 1) +EXCEPT ALL +SELECT * FROM bt_page_items('test4_a_1_idx', 1); +(0 rows) + -- Failure when using a non-btree index. CREATE INDEX test1_a_hash ON test1 USING hash(a); SELECT bt_metap('test1_a_hash'); diff --git a/contrib/pageinspect/sql/btree.sql b/contrib/pageinspect/sql/btree.sql index 102ebdefe3c..2670f85f79a 100644 --- a/contrib/pageinspect/sql/btree.sql +++ b/contrib/pageinspect/sql/btree.sql @@ -32,6 +32,16 @@ SELECT * FROM bt_page_items(get_raw_page('test1_a_idx', 0)); SELECT * FROM bt_page_items(get_raw_page('test1_a_idx', 1)); SELECT * FROM bt_page_items(get_raw_page('test1_a_idx', 2)); +---test index over virtual generated column +CREATE TABLE test4(a int, b int GENERATED ALWAYS AS (a + 1), c text); +INSERT INTO test4(a,c) VALUES (10,11), (10,11); +CREATE INDEX test4_b_idx ON test4 USING btree (b); +CREATE INDEX test4_a_1_idx ON test4 USING btree ((a+1)); +--expect return zero row +SELECT * FROM bt_page_items('test4_b_idx', 1) +EXCEPT ALL +SELECT * FROM bt_page_items('test4_a_1_idx', 1); + -- Failure when using a non-btree index. CREATE INDEX test1_a_hash ON test1 USING hash(a); SELECT bt_metap('test1_a_hash'); diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml index 2fc63442980..1405ac8424f 100644 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -4592,6 +4592,21 @@ SCRAM-SHA-256$<replaceable><iteration count></replaceable>:<replaceable>&l </para></entry> </row> + <row> + <entry role="catalog_table_entry"><para role="column_definition"> + <structfield>indattrgenerated</structfield> <type>int2vector</type> + (references <link linkend="catalog-pg-attribute"><structname>pg_attribute</structname></link>.<structfield>attnum</structfield>) + </para> + <para> + This is an array of <structfield>indnatts</structfield> values that + indicate which virtual generated columns this index indexes. + For example, a value of <literal>1 3</literal> would mean that the first + and the third table columns of this index entries are virtual generated + column. A zero in this array indicates that the corresponding index + attribute is not virtual generated column reference. + </para></entry> + </row> + <row> <entry role="catalog_table_entry"><para role="column_definition"> <structfield>indexprs</structfield> <type>pg_node_tree</type> diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c index 43de42ce39e..7b1d8e69c57 100644 --- a/src/backend/catalog/index.c +++ b/src/backend/catalog/index.c @@ -245,6 +245,16 @@ index_check_primary_key(Relation heapRel, HeapTuple atttuple; Form_pg_attribute attform; + /* + * Unique constraints and primary keys based on virtual generated + * columns are not supported. Nevertheless, it is still prudent to + * perform the check. + */ + if (indexInfo->ii_IndexAttrGeneratedNumbers[i] != 0) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("primary keys on virtual generated columns are not supported")); + if (attnum == 0) ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), @@ -582,6 +592,12 @@ UpdateIndexRelation(Oid indexoid, Relation pg_index; HeapTuple tuple; int i; + int2vector *indgenkey; + + int16 *colgenerated = palloc_array(int16, indexInfo->ii_NumIndexAttrs); + + for (i = 0; i < indexInfo->ii_NumIndexAttrs; i++) + colgenerated[i] = indexInfo->ii_IndexAttrGeneratedNumbers[i]; /* * Copy the index key, opclass, and indoption info into arrays (should we @@ -593,6 +609,7 @@ UpdateIndexRelation(Oid indexoid, indcollation = buildoidvector(collationOids, indexInfo->ii_NumIndexKeyAttrs); indclass = buildoidvector(opclassOids, indexInfo->ii_NumIndexKeyAttrs); indoption = buildint2vector(coloptions, indexInfo->ii_NumIndexKeyAttrs); + indgenkey = buildint2vector(colgenerated, indexInfo->ii_NumIndexAttrs); /* * Convert the index expressions (if any) to a text datum @@ -651,6 +668,7 @@ UpdateIndexRelation(Oid indexoid, values[Anum_pg_index_indcollation - 1] = PointerGetDatum(indcollation); values[Anum_pg_index_indclass - 1] = PointerGetDatum(indclass); values[Anum_pg_index_indoption - 1] = PointerGetDatum(indoption); + values[Anum_pg_index_indattrgenerated - 1] = PointerGetDatum(indgenkey); values[Anum_pg_index_indexprs - 1] = exprsDatum; if (exprsDatum == (Datum) 0) nulls[Anum_pg_index_indexprs - 1] = true; @@ -1132,6 +1150,28 @@ index_create(Relation heapRelation, } } + /* + * Internally, we convert index of virtual generation column into + * an expression index. For example, if column 'b' is defined as + * (b INT GENERATED ALWAYS AS (a * 2) VIRTUAL) then index over 'b' + * would transformed into an expression index as ((a * 2)). As a + * result, the pg_depend refobjsubid does not retain the original + * attribute number of the virtual generated column. But rebuild + * index need these original virtual generated column. Thus we + * need auto dependencies on referenced virtual generated columns. + */ + for (i = 0; i < indexInfo->ii_NumIndexAttrs; i++) + { + if (indexInfo->ii_IndexAttrGeneratedNumbers[i] != 0) + { + ObjectAddressSubSet(referenced, RelationRelationId, + heapRelationId, + indexInfo->ii_IndexAttrGeneratedNumbers[i]); + add_exact_object_address(&referenced, addrs); + have_simple_col = false; + } + } + /* * If there are no simply-referenced columns, give the index an * auto dependency on the whole table. In most cases, this will @@ -1409,6 +1449,7 @@ index_concurrently_create_copy(Relation heapRelation, Oid oldIndexId, indexColNames = lappend(indexColNames, NameStr(att->attname)); newInfo->ii_IndexAttrNumbers[i] = oldInfo->ii_IndexAttrNumbers[i]; + newInfo->ii_IndexAttrGeneratedNumbers[i] = oldInfo->ii_IndexAttrGeneratedNumbers[i]; } /* Extract opclass options for each attribute */ @@ -2426,9 +2467,12 @@ IndexInfo * BuildIndexInfo(Relation index) { IndexInfo *ii; + HeapTuple ht_idx; Form_pg_index indexStruct = index->rd_index; int i; int numAtts; + Datum indgenkeyDatum; + int2vector *indgenkey; /* check the number of keys, and copy attr numbers into the IndexInfo */ numAtts = indexStruct->indnatts; @@ -2452,9 +2496,18 @@ BuildIndexInfo(Relation index) index->rd_indam->amsummarizing, indexStruct->indisexclusion && indexStruct->indisunique); + ht_idx = SearchSysCache1(INDEXRELID, ObjectIdGetDatum(indexStruct->indexrelid)); + indgenkeyDatum = SysCacheGetAttrNotNull(INDEXRELID, ht_idx, + Anum_pg_index_indattrgenerated); + indgenkey = (int2vector *) DatumGetPointer(indgenkeyDatum); + /* fill in attribute numbers */ for (i = 0; i < numAtts; i++) + { ii->ii_IndexAttrNumbers[i] = indexStruct->indkey.values[i]; + ii->ii_IndexAttrGeneratedNumbers[i] = indgenkey->values[i]; + } + ReleaseSysCache(ht_idx); /* fetch exclusion constraint info if any */ if (indexStruct->indisexclusion) @@ -2516,11 +2569,34 @@ BuildDummyIndexInfo(Relation index) for (i = 0; i < numAtts; i++) ii->ii_IndexAttrNumbers[i] = indexStruct->indkey.values[i]; + /* + * Index expressions or predicates are skipped here, see above comments. + * If virtual generated columns references another column, + * ii_IndexAttrNumbers will set to that referenced column. So do nothing + * for ii_IndexAttrGeneratedNumbers here. + */ + /* We ignore the exclusion constraint if any */ return ii; } +/* + * indexContainVirtual + * Return true if this index contains a reference to a virtual generated + * column. + */ +bool +indexContainVirtual(const IndexInfo *info) +{ + for (int i = 0; i < info->ii_NumIndexAttrs; i++) + { + if (AttributeNumberIsValid(info->ii_IndexAttrGeneratedNumbers[i])) + return true; + } + return false; +} + /* * CompareIndexInfo * Return whether the properties of two indexes (in different tables) @@ -2568,6 +2644,15 @@ CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2, if (attmap->maplen < info2->ii_IndexAttrNumbers[i]) elog(ERROR, "incorrect attribute map"); + if (AttributeNumberIsValid(info1->ii_IndexAttrGeneratedNumbers[i]) || + AttributeNumberIsValid(info2->ii_IndexAttrGeneratedNumbers[i])) + { + /* fail if index over virtual generated column does not match */ + if (attmap->attnums[info2->ii_IndexAttrGeneratedNumbers[i] - 1] != + info1->ii_IndexAttrGeneratedNumbers[i]) + return false; + } + /* ignore expressions for now (but check their collation/opfamily) */ if (!(info1->ii_IndexAttrNumbers[i] == InvalidAttrNumber && info2->ii_IndexAttrNumbers[i] == InvalidAttrNumber)) diff --git a/src/backend/commands/indexcmds.c b/src/backend/commands/indexcmds.c index 755dc00c86f..5b677483be2 100644 --- a/src/backend/commands/indexcmds.c +++ b/src/backend/commands/indexcmds.c @@ -53,6 +53,7 @@ #include "parser/parse_utilcmd.h" #include "partitioning/partdesc.h" #include "pgstat.h" +#include "rewrite/rewriteHandler.h" #include "rewrite/rewriteManip.h" #include "storage/lmgr.h" #include "storage/proc.h" @@ -90,9 +91,17 @@ static void ComputeIndexAttrs(ParseState *pstate, bool amcanorder, bool isconstraint, bool iswithoutoverlaps, + bool is_primary, Oid ddl_userid, int ddl_sec_context, int *ddl_save_nestlevel); +static void ComputeIndexGeneratedAttrs(ParseState *pstate, + IndexInfo *indexInfo, + Relation rel, + bool is_primary, + int attn, + int attnum, + int location); static char *ChooseIndexName(const char *tabname, Oid namespaceId, const List *colnames, const List *exclusionOpNames, bool primary, bool isconstraint); @@ -182,6 +191,7 @@ CheckIndexCompatible(Oid oldId, bool isWithoutOverlaps) { bool isconstraint; + bool is_primary; Oid *typeIds; Oid *collationIds; Oid *opclassIds; @@ -214,6 +224,12 @@ CheckIndexCompatible(Oid oldId, */ isconstraint = false; + /* + * We can pretend is_primary = false unconditionally. It only serves to + * decide the text of an error message that should never happen for us. + */ + is_primary = false; + numberOfAttributes = list_length(attributeList); Assert(numberOfAttributes > 0); Assert(numberOfAttributes <= INDEX_MAX_KEYS); @@ -254,7 +270,7 @@ CheckIndexCompatible(Oid oldId, coloptions, attributeList, exclusionOpNames, relationId, accessMethodName, accessMethodId, - amcanorder, isconstraint, isWithoutOverlaps, InvalidOid, + amcanorder, isconstraint, isWithoutOverlaps, is_primary, InvalidOid, 0, NULL); /* Get the soon-obsolete pg_index tuple. */ @@ -907,6 +923,27 @@ DefineIndex(ParseState *pstate, if (stmt->whereClause) CheckPredicate((Expr *) stmt->whereClause); + /* virtual generated column over predicate indexes are not supported */ + if (RelationGetDescr(rel)->constr && + RelationGetDescr(rel)->constr->has_generated_virtual && + stmt->whereClause) + { + Bitmapset *indexattrs_pred = NULL; + int j = -1; + + pull_varattnos(stmt->whereClause, 1, &indexattrs_pred); + + while ((j = bms_next_member(indexattrs_pred, j)) >= 0) + { + AttrNumber attno = j + FirstLowInvalidHeapAttributeNumber; + + if (TupleDescAttr(RelationGetDescr(rel), attno - 1)->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("partial index on virtual generated columns are not supported")); + } + } + /* * Parse AM-specific options, convert to text array form, validate. */ @@ -944,6 +981,7 @@ DefineIndex(ParseState *pstate, stmt->excludeOpNames, tableId, accessMethodName, accessMethodId, amcanorder, stmt->isconstraint, stmt->iswithoutoverlaps, + stmt->primary, root_save_userid, root_save_sec_context, &root_save_nestlevel); @@ -1025,6 +1063,11 @@ DefineIndex(ParseState *pstate, constraint_type))); /* Search the index column(s) for a match */ + + /* + * Unique constraints and primary keys based on virtual generated + * columns are not supported. So no need worry about it here. + */ for (j = 0; j < indexInfo->ii_NumIndexKeyAttrs; j++) { if (key->partattrs[i] == indexInfo->ii_IndexAttrNumbers[j]) @@ -1105,9 +1148,6 @@ DefineIndex(ParseState *pstate, /* * We disallow indexes on system columns. They would not necessarily get * updated correctly, and they don't seem useful anyway. - * - * Also disallow virtual generated columns in indexes (use expression - * index instead). */ for (int i = 0; i < indexInfo->ii_NumIndexAttrs; i++) { @@ -1117,26 +1157,14 @@ DefineIndex(ParseState *pstate, ereport(ERROR, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("index creation on system columns is not supported"))); - - - if (TupleDescAttr(RelationGetDescr(rel), attno - 1)->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL) - ereport(ERROR, - errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - stmt->primary ? - errmsg("primary keys on virtual generated columns are not supported") : - stmt->isconstraint ? - errmsg("unique constraints on virtual generated columns are not supported") : - errmsg("indexes on virtual generated columns are not supported")); } /* - * Also check for system and generated columns used in expressions or - * predicates. + * Also check for system columns used in expressions or predicates. */ if (indexInfo->ii_Expressions || indexInfo->ii_Predicate) { Bitmapset *indexattrs = NULL; - int j; pull_varattnos((Node *) indexInfo->ii_Expressions, 1, &indexattrs); pull_varattnos((Node *) indexInfo->ii_Predicate, 1, &indexattrs); @@ -1149,24 +1177,6 @@ DefineIndex(ParseState *pstate, (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), errmsg("index creation on system columns is not supported"))); } - - /* - * XXX Virtual generated columns in index expressions or predicates - * could be supported, but it needs support in - * RelationGetIndexExpressions() and RelationGetIndexPredicate(). - */ - j = -1; - while ((j = bms_next_member(indexattrs, j)) >= 0) - { - AttrNumber attno = j + FirstLowInvalidHeapAttributeNumber; - - if (TupleDescAttr(RelationGetDescr(rel), attno - 1)->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL) - ereport(ERROR, - (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), - stmt->isconstraint ? - errmsg("unique constraints on virtual generated columns are not supported") : - errmsg("indexes on virtual generated columns are not supported"))); - } } /* Is index safe for others to ignore? See set_indexsafe_procflags() */ @@ -1310,6 +1320,7 @@ DefineIndex(ParseState *pstate, bool invalidate_parent = false; Relation parentIndex; TupleDesc parentDesc; + bool parent_idx_virtual; /* * Report the total number of partitions at the start of the @@ -1355,6 +1366,7 @@ DefineIndex(ParseState *pstate, */ parentIndex = index_open(indexRelationId, lockmode); indexInfo = BuildIndexInfo(parentIndex); + parent_idx_virtual = indexContainVirtual(indexInfo); parentDesc = RelationGetDescr(rel); @@ -1415,6 +1427,15 @@ DefineIndex(ParseState *pstate, parentDesc, false); + /* + * If parent has an index on a virtual generated column, we + * must ensure that the indexed generated expression on the + * parent matches that of the child. + */ + if (parent_idx_virtual) + check_generated_indexattrs(indexInfo, rel, + childrel, attmap, false); + foreach(cell, childidxs) { Oid cldidxid = lfirst_oid(cell); @@ -1862,6 +1883,66 @@ CheckPredicate(Expr *predicate) errmsg("functions in index predicate must be marked IMMUTABLE"))); } +/* + * Verify that the generated expression of the parent matches the child + * + * indexinfo: the IndexInfo that is associated with relation "rel". + * childrel : the relation to be attached to "rel" or the child of "rel". + * attmap : Attribute mapping between childrel and rel. + * is_attach: is this command of ALTER TABLE ATTACH PARTITION + * + * Use build_attrmap_by_name(childrel, rel) to build the attmap. +*/ +void +check_generated_indexattrs(const IndexInfo *indexinfo, + Relation rel, + Relation childrel, + const AttrMap *attmap, + bool is_attach) +{ + /* if parent have virtual generated column, child must also have */ + Assert(rel->rd_att->constr->has_generated_virtual); + Assert(childrel->rd_att->constr->has_generated_virtual); + + for (int i = 0; i < indexinfo->ii_NumIndexAttrs; i++) + { + if (AttributeNumberIsValid(indexinfo->ii_IndexAttrGeneratedNumbers[i])) + { + Node *node_parent; + Node *node_child; + bool found_whole_row; + + AttrNumber attno = indexinfo->ii_IndexAttrGeneratedNumbers[i]; + + node_parent = build_generation_expression(rel, attno); + + node_parent = map_variable_attnos(node_parent, + 1, + 0, + attmap, + InvalidOid, &found_whole_row); + if (found_whole_row) + elog(ERROR, "Index contains a whole-row table reference"); + + node_child = build_generation_expression(childrel, + attmap->attnums[attno - 1]); + + if (!equal(node_parent, node_child)) + ereport(ERROR, + errcode(ERRCODE_WRONG_OBJECT_TYPE), + is_attach ? + errmsg("cannot attach table \"%s\" as partition of partitioned table \"%s\"", + RelationGetRelationName(childrel), + RelationGetRelationName(rel)) : + errmsg("cannot create index on partitioned table \"%s\"", + RelationGetRelationName(rel)), + errdetail("The virtual generated column expression of partitioned table \"%s\" does not match that of table \"%s\"", + RelationGetRelationName(rel), + RelationGetRelationName(childrel))); + } + } +} + /* * Compute per-index-column information, including indexed column numbers * or index expressions, opclasses and their options. Note, all output vectors @@ -1887,6 +1968,7 @@ ComputeIndexAttrs(ParseState *pstate, bool amcanorder, bool isconstraint, bool iswithoutoverlaps, + bool is_primary, Oid ddl_userid, int ddl_sec_context, int *ddl_save_nestlevel) @@ -1897,6 +1979,25 @@ ComputeIndexAttrs(ParseState *pstate, int nkeycols = indexInfo->ii_NumIndexKeyAttrs; Oid save_userid; int save_sec_context; + List *virtual_generated = NIL; + Relation rel = table_open(relId, NoLock); + TupleDesc reltupldesc = RelationGetDescr(rel); + + /* + * Virtual generated columns are not currently supported in expression + * indexes. We therefore collect the virtual generated columns attribute + * number for subsequent verification of expression attributes. + */ + if (reltupldesc->constr && reltupldesc->constr->has_generated_virtual) + { + for (int i = 0; i < reltupldesc->natts; i++) + { + Form_pg_attribute attr = TupleDescAttr(reltupldesc, i); + + if (!attr->attisdropped && attr->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL) + virtual_generated = lappend_int(virtual_generated, attr->attnum); + } + } /* Allocate space for exclusion operator info, if needed */ if (exclusionOpNames) @@ -1968,7 +2069,19 @@ ComputeIndexAttrs(ParseState *pstate, parser_errposition(pstate, attribute->location))); } attform = (Form_pg_attribute) GETSTRUCT(atttuple); - indexInfo->ii_IndexAttrNumbers[attn] = attform->attnum; + + if (attform->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL) + ComputeIndexGeneratedAttrs(pstate, indexInfo, rel, + is_primary, + attn, + attform->attnum, + attribute->location); + else + { + indexInfo->ii_IndexAttrGeneratedNumbers[attn] = 0; + indexInfo->ii_IndexAttrNumbers[attn] = attform->attnum; + } + atttype = attform->atttypid; attcollation = attform->attcollation; ReleaseSysCache(atttuple); @@ -1995,18 +2108,47 @@ ComputeIndexAttrs(ParseState *pstate, while (IsA(expr, CollateExpr)) expr = (Node *) ((CollateExpr *) expr)->arg; + if (!IsA(expr, Var)) + { + Bitmapset *idxattrs = NULL; + int j = -1; + + pull_varattnos(expr, 1, &idxattrs); + while ((j = bms_next_member(idxattrs, j)) >= 0) + { + if (list_member_int(virtual_generated, j + FirstLowInvalidHeapAttributeNumber)) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("expression index over virtual generated columns are not supported"), + parser_errposition(pstate, attribute->location)); + } + } + if (IsA(expr, Var) && ((Var *) expr)->varattno != InvalidAttrNumber) { - /* - * User wrote "(column)" or "(column COLLATE something)". - * Treat it like simple attribute anyway. - */ - indexInfo->ii_IndexAttrNumbers[attn] = ((Var *) expr)->varattno; + int attnum = ((Var *) expr)->varattno; + + if (list_member_int(virtual_generated, attnum)) + ComputeIndexGeneratedAttrs(pstate, indexInfo, rel, + is_primary, + attn, + attnum, + attribute->location); + else + { + /* + * User wrote "(column)" or "(column COLLATE something)". + * Treat it like simple attribute anyway. + */ + indexInfo->ii_IndexAttrNumbers[attn] = attnum; + indexInfo->ii_IndexAttrGeneratedNumbers[attn] = 0; + } } else { indexInfo->ii_IndexAttrNumbers[attn] = 0; /* marks expression */ + indexInfo->ii_IndexAttrGeneratedNumbers[attn] = 0; indexInfo->ii_Expressions = lappend(indexInfo->ii_Expressions, expr); @@ -2268,6 +2410,92 @@ ComputeIndexAttrs(ParseState *pstate, attn++; } + + table_close(rel, NoLock); +} + +/* + * Compute IndexInfo fields: ii_IndexAttrNumbers, ii_IndexAttrGeneratedNumbers, + * and ii_Expressions. + * + * pstate: ParseState struct (used only for error reports; pass NULL if not available) + * indexInfo: the IndexInfo structure to be built. + * rel: the relation on which this IndexInfo is based. + * is_primary: indicates whether this index is a primary key. + * attn: zero-based indexes of the index key attributes. + * attnum: attribute number of the virtual generated column. + * location: index key location. + */ +static void +ComputeIndexGeneratedAttrs(ParseState *pstate, + IndexInfo *indexInfo, Relation rel, + bool is_primary, int attn, int attnum, + int location) +{ + Node *node; + + if (is_primary) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("primary keys on virtual generated columns are not supported"), + parser_errposition(pstate, location)); + + if (indexInfo->ii_Unique) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("unique constraints on virtual generated columns are not supported"), + parser_errposition(pstate, location)); + + if (attn >= indexInfo->ii_NumIndexKeyAttrs) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("virtual generated column are not supported in index included columns"), + parser_errposition(pstate, location)); + + /* Fetch the GENERATED AS expression tree */ + node = build_generation_expression(rel, attnum); + + /* + * if the generation expression just reference another column, then set + * ii_IndexAttrNumbers to that column attribute number. + */ + if (IsA(node, Var)) + { + Var *var = (Var *) node; + + if (var->varattno < 0) + ereport(ERROR, + errcode(ERRCODE_FEATURE_NOT_SUPPORTED), + errmsg("index creation on system columns is not supported"), + parser_errposition(pstate, location)); + + indexInfo->ii_IndexAttrNumbers[attn] = var->varattno; + } + else + { + /* + * Strip any top-level COLLATE clause in generated expression. This + * ensures that we treat "x COLLATE y" and "(x COLLATE y)" alike. + */ + while (IsA(node, CollateExpr)) + node = (Node *) ((CollateExpr *) node)->arg; + + if (IsA(node, Var)) + { + Var *var = (Var *) node; + + Assert(var->varattno > 0); + indexInfo->ii_IndexAttrNumbers[attn] = var->varattno; + } + else + { + /* mark as expression index */ + indexInfo->ii_IndexAttrNumbers[attn] = 0; + indexInfo->ii_Expressions = lappend(indexInfo->ii_Expressions, + node); + } + } + indexInfo->ii_IndexAttrGeneratedNumbers[attn] = attnum; } /* diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c index f976c0e5c7e..6cc296b2957 100644 --- a/src/backend/commands/tablecmds.c +++ b/src/backend/commands/tablecmds.c @@ -8699,6 +8699,21 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, ReleaseSysCache(tuple); + /* + * Find everything that depends on the column (constraints, indexes, etc), + * and record enough information to let us recreate the objects after + * rewrite. + */ + RememberAllDependentForRebuilding(tab, AT_SetExpression, rel, attnum, colName); + + /* + * Changing virtual geneeration expression does not require table rewrite. + * However, if any index is built on top of it, table rewrite is + * necessary. + */ + if (tab->changedIndexOids != NIL) + rewrite = true; + if (rewrite) { /* @@ -8709,13 +8724,6 @@ ATExecSetExpression(AlteredTableInfo *tab, Relation rel, const char *colName, /* make sure we don't conflict with later attribute modifications */ CommandCounterIncrement(); - - /* - * Find everything that depends on the column (constraints, indexes, - * etc), and record enough information to let us recreate the objects - * after rewrite. - */ - RememberAllDependentForRebuilding(tab, AT_SetExpression, rel, attnum, colName); } /* @@ -14870,6 +14878,27 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel, */ RememberAllDependentForRebuilding(tab, AT_AlterColumnType, rel, attnum, colName); + /* + * Tell phase3 do table rewrite if there are any index based on virtual + * generated colum. + */ + if (attTup->attgenerated == ATTRIBUTE_GENERATED_VIRTUAL && + tab->changedIndexOids != NIL) + { + Relation newrel; + + newrel = table_open(RelationGetRelid(rel), NoLock); + + RelationClearMissing(newrel); + + relation_close(newrel, NoLock); + + /* make sure we don't conflict with later attribute modifications */ + CommandCounterIncrement(); + + tab->rewrite |= AT_REWRITE_COLUMN_REWRITE; + } + /* * Now scan for dependencies of this column on other things. The only * things we should find are the dependency on the column datatype and @@ -20699,6 +20728,10 @@ AttachPartitionEnsureIndexes(List **wqueue, Relation rel, Relation attachrel) attmap = build_attrmap_by_name(RelationGetDescr(attachrel), RelationGetDescr(rel), false); + + if (indexContainVirtual(info)) + check_generated_indexattrs(info, rel, attachrel, attmap, true); + constraintOid = get_relation_idx_constraint_oid(RelationGetRelid(rel), idx); /* diff --git a/src/backend/executor/execIndexing.c b/src/backend/executor/execIndexing.c index 6ae0f959592..06329347a30 100644 --- a/src/backend/executor/execIndexing.c +++ b/src/backend/executor/execIndexing.c @@ -1039,6 +1039,18 @@ index_unchanged_by_update(ResultRelInfo *resultRelInfo, EState *estate, for (int attr = 0; attr < indexInfo->ii_NumIndexKeyAttrs; attr++) { int keycol = indexInfo->ii_IndexAttrNumbers[attr]; + int vkeycol = indexInfo->ii_IndexAttrGeneratedNumbers[attr]; + + if (vkeycol > 0 && + (bms_is_member(vkeycol - FirstLowInvalidHeapAttributeNumber, + updatedCols) || + bms_is_member(vkeycol - FirstLowInvalidHeapAttributeNumber, + extraUpdatedCols))) + { + /* Changed key column -- don't hint for this index */ + indexInfo->ii_IndexUnchanged = false; + return false; + } if (keycol <= 0) { diff --git a/src/backend/executor/execPartition.c b/src/backend/executor/execPartition.c index 55bdf5c4835..439d85fedcb 100644 --- a/src/backend/executor/execPartition.c +++ b/src/backend/executor/execPartition.c @@ -538,6 +538,11 @@ IsIndexCompatibleAsArbiter(Relation arbiterIndexRelation, if (arbiterIndexRelation->rd_index->indkey.values[i] != indexRelation->rd_index->indkey.values[i]) return false; + + /* + * Unique indexes on virtual generated columns are not supported, no + * need to worry about indattrgenerated here. + */ } if (list_difference(RelationGetIndexExpressions(arbiterIndexRelation), diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c index 652f7538c37..48fd08e428d 100644 --- a/src/backend/parser/parse_utilcmd.c +++ b/src/backend/parser/parse_utilcmd.c @@ -1705,6 +1705,7 @@ generateClonedIndexStmt(RangeVar *heapRel, Relation source_idx, Form_pg_am amrec; oidvector *indcollation; oidvector *indclass; + int2vector *indgenkey; IndexStmt *index; List *indexprs; ListCell *indexpr_item; @@ -1712,6 +1713,7 @@ generateClonedIndexStmt(RangeVar *heapRel, Relation source_idx, int keyno; Oid keycoltype; Datum datum; + Datum indgenkeyDatum; bool isnull; if (constraintOid) @@ -1747,6 +1749,11 @@ generateClonedIndexStmt(RangeVar *heapRel, Relation source_idx, datum = SysCacheGetAttrNotNull(INDEXRELID, ht_idx, Anum_pg_index_indclass); indclass = (oidvector *) DatumGetPointer(datum); + /* Extract indattrgenerated from the pg_index tuple */ + indgenkeyDatum = SysCacheGetAttrNotNull(INDEXRELID, ht_idx, + Anum_pg_index_indattrgenerated); + indgenkey = (int2vector *) DatumGetPointer(indgenkeyDatum); + /* Begin building the IndexStmt */ index = makeNode(IndexStmt); index->relation = heapRel; @@ -1878,13 +1885,26 @@ generateClonedIndexStmt(RangeVar *heapRel, Relation source_idx, { IndexElem *iparam; AttrNumber attnum = idxrec->indkey.values[keyno]; + AttrNumber gennum = indgenkey->values[keyno]; Form_pg_attribute attr = TupleDescAttr(RelationGetDescr(source_idx), keyno); int16 opt = source_idx->rd_indoption[keyno]; iparam = makeNode(IndexElem); - if (AttributeNumberIsValid(attnum)) + if (AttributeNumberIsValid(gennum)) + { + /* + * An index on a virtual generated column was converted into an + * expression index, but here we need the original attribute + * number. + */ + keycoltype = get_atttype(indrelid, gennum); + + iparam->name = get_attname(indrelid, gennum, false);; + iparam->expr = NULL; + } + else if (AttributeNumberIsValid(attnum)) { /* Simple index column */ char *attname; @@ -1977,6 +1997,10 @@ generateClonedIndexStmt(RangeVar *heapRel, Relation source_idx, iparam = makeNode(IndexElem); + /* + * INCLUDED COLUMNS based on virtual generated columns are not + * supported. So no need worry about it here. + */ if (AttributeNumberIsValid(attnum)) { /* Simple index column */ @@ -2528,6 +2552,9 @@ transformIndexConstraint(Constraint *constraint, CreateStmtContext *cxt) * We shouldn't see attnum == 0 here, since we already rejected * expression indexes. If we do, SystemAttributeDefinition will * throw an error. + * + * Unique constraints and primary keys based on virtual generated + * columns are not supported. So no need worry about it here. */ if (attnum > 0) { diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c index 416f1a21ae4..2fd1b334a08 100644 --- a/src/backend/utils/adt/ruleutils.c +++ b/src/backend/utils/adt/ruleutils.c @@ -1290,9 +1290,11 @@ pg_get_indexdef_worker(Oid indexrelid, int colno, Datum indcollDatum; Datum indclassDatum; Datum indoptionDatum; + Datum indgenkeyDatum; oidvector *indcollation; oidvector *indclass; int2vector *indoption; + int2vector *indgenkey; StringInfoData buf; char *str; char *sep; @@ -1325,6 +1327,10 @@ pg_get_indexdef_worker(Oid indexrelid, int colno, Anum_pg_index_indoption); indoption = (int2vector *) DatumGetPointer(indoptionDatum); + indgenkeyDatum = SysCacheGetAttrNotNull(INDEXRELID, ht_idx, + Anum_pg_index_indattrgenerated); + indgenkey = (int2vector *) DatumGetPointer(indgenkeyDatum); + /* * Fetch the pg_class tuple of the index relation */ @@ -1398,6 +1404,7 @@ pg_get_indexdef_worker(Oid indexrelid, int colno, for (keyno = 0; keyno < idxrec->indnatts; keyno++) { AttrNumber attnum = idxrec->indkey.values[keyno]; + AttrNumber gennum = indgenkey->values[keyno]; Oid keycoltype; Oid keycolcollation; @@ -1418,7 +1425,32 @@ pg_get_indexdef_worker(Oid indexrelid, int colno, appendStringInfoString(&buf, sep); sep = ", "; - if (attnum != 0) + /* + * We must first check indexes on virtual generated columns, and then + * simple index columns, since a virtual generated column can + * reference another regular column. + */ + if (AttributeNumberIsValid(gennum)) + { + char *virtual_attname; + int32 geneycoltypmod; + + virtual_attname = get_attname(indrelid, gennum, false); + if (!colno || colno == keyno + 1) + appendStringInfoString(&buf, quote_identifier(virtual_attname)); + + get_atttypetypmodcoll(indrelid, gennum, + &keycoltype, &geneycoltypmod, + &keycolcollation); + + /* + * The virtual generated column has already been printed, so its + * generation expression should not be emitted again. + */ + if (!AttributeNumberIsValid(attnum)) + indexpr_item = lnext(indexprs, indexpr_item); + } + else if (attnum != 0) { /* Simple index column */ char *attname; diff --git a/src/include/catalog/index.h b/src/include/catalog/index.h index b259c4141ed..e9519a3cafd 100644 --- a/src/include/catalog/index.h +++ b/src/include/catalog/index.h @@ -126,6 +126,7 @@ extern IndexInfo *BuildIndexInfo(Relation index); extern IndexInfo *BuildDummyIndexInfo(Relation index); +extern bool indexContainVirtual(const IndexInfo *info); extern bool CompareIndexInfo(const IndexInfo *info1, const IndexInfo *info2, const Oid *collations1, const Oid *collations2, const Oid *opfamilies1, const Oid *opfamilies2, @@ -175,6 +176,11 @@ extern void RestoreReindexState(const void *reindexstate); extern void IndexSetParentIndex(Relation partitionIdx, Oid parentOid); +extern void check_generated_indexattrs(const IndexInfo *rel_idx_info, + Relation rel, + Relation childrel, + const AttrMap *attmap, + bool is_attach); /* * itemptr_encode - Encode ItemPointer as int64/int8 diff --git a/src/include/catalog/pg_index.h b/src/include/catalog/pg_index.h index 02c99d70faf..e3097106d8e 100644 --- a/src/include/catalog/pg_index.h +++ b/src/include/catalog/pg_index.h @@ -54,6 +54,9 @@ CATALOG(pg_index,2610,IndexRelationId) BKI_SCHEMA_MACRO oidvector indclass BKI_LOOKUP(pg_opclass) BKI_FORCE_NOT_NULL; /* opclass identifiers */ int2vector indoption BKI_FORCE_NOT_NULL; /* per-column flags * (AM-specific meanings) */ + int2vector indattrgenerated BKI_FORCE_NOT_NULL; /* the attribute of + * virtual generated + * column? */ pg_node_tree indexprs; /* expression trees for index attributes that * are not simple column references; one for * each zero entry in indkey[] */ diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h index 02265456978..58079e49f7a 100644 --- a/src/include/nodes/execnodes.h +++ b/src/include/nodes/execnodes.h @@ -174,6 +174,14 @@ typedef struct IndexInfo */ AttrNumber ii_IndexAttrNumbers[INDEX_MAX_KEYS]; + /* + * Underlying-rel virtual generated attribute numbers used as keys. A + * value of zero indicates that the index key does not reference a virtual + * generated column. Note: Virtual generated columns cannot be used in + * INCLUDED index columns. + */ + AttrNumber ii_IndexAttrGeneratedNumbers[INDEX_MAX_KEYS]; + /* expr trees for expression entries, or NIL if none */ List *ii_Expressions; /* list of Expr */ /* exec state for expressions, or NIL if none */ diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out index ac1a7345d0f..7e530357345 100644 --- a/src/test/regress/expected/alter_table.out +++ b/src/test/regress/expected/alter_table.out @@ -119,6 +119,26 @@ HINT: Alter statistics on table column instead. ALTER INDEX attmp_idx ALTER COLUMN 4 SET STATISTICS 1000; ERROR: column number 4 of relation "attmp_idx" does not exist ALTER INDEX attmp_idx ALTER COLUMN 2 SET STATISTICS -1; +ALTER TABLE attmp + ADD COLUMN col1 int GENERATED ALWAYS AS (a), + ADD COLUMN col2 int GENERATED ALWAYS AS (a + 1); +CREATE INDEX attmp_idx1 ON attmp (a, col1, col2); +ALTER INDEX attmp_idx1 ALTER COLUMN 1 SET STATISTICS 1000; +ERROR: cannot alter statistics on non-expression column "a" of index "attmp_idx1" +HINT: Alter statistics on table column instead. +ALTER INDEX attmp_idx1 ALTER COLUMN 2 SET STATISTICS 1000; +ERROR: cannot alter statistics on non-expression column "col1" of index "attmp_idx1" +HINT: Alter statistics on table column instead. +ALTER INDEX attmp_idx1 ALTER COLUMN 3 SET STATISTICS 1000; +\d+ attmp_idx1 + Index "public.attmp_idx1" + Column | Type | Key? | Definition | Storage | Stats target +--------+---------+------+------------+---------+-------------- + a | integer | yes | a | plain | + col1 | integer | yes | col1 | plain | + col2 | integer | yes | col2 | plain | 1000 +btree, for table "public.attmp" + DROP TABLE attmp; -- -- rename - check on both non-temp and temp tables diff --git a/src/test/regress/expected/collate.icu.utf8.out b/src/test/regress/expected/collate.icu.utf8.out index 1325e123877..c1ad91b23e5 100644 --- a/src/test/regress/expected/collate.icu.utf8.out +++ b/src/test/regress/expected/collate.icu.utf8.out @@ -2703,6 +2703,19 @@ SELECT * FROM t5 ORDER BY c ASC, a ASC; 3 | d1 | d1 (3 rows) +CREATE INDEX t5_idx1 ON t5 USING btree((b COLLATE "POSIX")); +CREATE INDEX t5_idx2 ON t5 USING btree((b)); +SELECT indexrelid::regclass, indrelid::regclass, indnatts, indattrgenerated, + indcollation[0]::regcollation +FROM pg_index +WHERE indrelid = 't5'::regclass +ORDER BY indexrelid; + indexrelid | indrelid | indnatts | indattrgenerated | indcollation +------------+----------+----------+------------------+-------------- + t5_idx1 | t5 | 1 | 0 | "POSIX" + t5_idx2 | t5 | 1 | 0 | "C" +(2 rows) + -- Check that DEFAULT expressions in SQL/JSON functions use the same collation -- as the RETURNING type. Mismatched collations should raise an error. CREATE DOMAIN d1 AS text COLLATE case_insensitive; diff --git a/src/test/regress/expected/fast_default.out b/src/test/regress/expected/fast_default.out index ccbcdf8403f..ef19f667cc1 100644 --- a/src/test/regress/expected/fast_default.out +++ b/src/test/regress/expected/fast_default.out @@ -70,6 +70,14 @@ NOTICE: rewriting table has_volatile for reason 4 -- stored generated columns need a rewrite ALTER TABLE has_volatile ADD col7 int GENERATED ALWAYS AS (55) stored; NOTICE: rewriting table has_volatile for reason 2 +-- if there is any index over virtual generated columns, +-- change generation expression need rewrite +CREATE INDEX on has_volatile(col6); +ALTER TABLE has_volatile ALTER COLUMN col6 SET EXPRESSION AS (col1 * 3); +NOTICE: rewriting table has_volatile for reason 2 +-- table rewrite again. +ALTER TABLE has_volatile ALTER COLUMN col6 SET DATA TYPE INT8; +NOTICE: rewriting table has_volatile for reason 4 -- Test a large sample of different datatypes CREATE TABLE T(pk INT NOT NULL PRIMARY KEY, c_int INT DEFAULT 1); SELECT set('t'); diff --git a/src/test/regress/expected/generated_virtual.out b/src/test/regress/expected/generated_virtual.out index 249e68be654..147c6980540 100644 --- a/src/test/regress/expected/generated_virtual.out +++ b/src/test/regress/expected/generated_virtual.out @@ -745,30 +745,322 @@ ERROR: primary keys on virtual generated columns are not supported --INSERT INTO gtest22b VALUES (2); --INSERT INTO gtest22b VALUES (2); -- indexes -CREATE TABLE gtest22c (a int, b int GENERATED ALWAYS AS (a * 2) VIRTUAL); ---CREATE INDEX gtest22c_b_idx ON gtest22c (b); +CREATE TABLE gtestparted (b integer, c integer, a integer GENERATED ALWAYS AS (c+1)) PARTITION BY RANGE (b); +CREATE TABLE gtestpart3 (a integer GENERATED ALWAYS AS (c), b integer, c integer); +ALTER TABLE gtestparted ATTACH PARTITION gtestpart3 FOR VALUES FROM (1) to (10); +-- error: An index cannot be created when the partitioned table and its +-- partitions have different generation expressions. +CREATE INDEX gtestparted_a_idx_error on gtestparted(a); +ERROR: cannot create index on partitioned table "gtestparted" +DETAIL: The virtual generated column expression of partitioned table "gtestparted" does not match that of table "gtestpart3" +ALTER TABLE gtestpart3 ALTER COLUMN a SET EXPRESSION AS (c + 1); +CREATE TABLE gtestpart1 (c integer, a integer GENERATED ALWAYS AS (c), b integer); +CREATE TABLE gtestpart2 (b integer, a integer GENERATED ALWAYS AS (c+1), c integer); +CREATE INDEX gtestpart1_a_idx on gtestpart1(a); +CREATE INDEX gtestpart2_a_idx on gtestpart2(a); +CREATE INDEX gtestpart2_a_idx_copy on gtestpart2(a); +CREATE INDEX gtestparted_a_idx on gtestparted(a); +-- error: An index cannot be created when the partitioned table and its +-- partitions have different generation expressions. +ALTER TABLE gtestparted ATTACH PARTITION gtestpart1 FOR VALUES FROM (10) TO (20); +ERROR: cannot attach table "gtestpart1" as partition of partitioned table "gtestparted" +DETAIL: The virtual generated column expression of partitioned table "gtestparted" does not match that of table "gtestpart1" +ALTER TABLE gtestparted ATTACH PARTITION gtestpart2 FOR VALUES FROM (10) TO (20); --ok +SELECT * FROM pg_partition_tree('gtestparted_a_idx'::regclass); + relid | parentrelid | isleaf | level +-------------------+-------------------+--------+------- + gtestparted_a_idx | | f | 0 + gtestpart2_a_idx | gtestparted_a_idx | t | 1 + gtestpart3_a_idx | gtestparted_a_idx | t | 1 +(3 rows) + +ALTER INDEX gtestparted_a_idx ATTACH PARTITION gtestpart2_a_idx_copy; --error +ERROR: cannot attach index "gtestpart2_a_idx_copy" as a partition of index "gtestparted_a_idx" +DETAIL: Another index "gtestpart2_a_idx" is already attached for partition "gtestpart2". +ALTER INDEX gtestparted_a_idx ATTACH PARTITION gtestpart2_a_idx; --ok +CREATE INDEX gtestparted_a_idx_1 on gtestparted(a); +--now index gtestpart2_a_idx_copy should attach to the partition tree +SELECT * FROM pg_partition_tree('gtestparted_a_idx_1'::regclass); + relid | parentrelid | isleaf | level +-----------------------+---------------------+--------+------- + gtestparted_a_idx_1 | | f | 0 + gtestpart2_a_idx_copy | gtestparted_a_idx_1 | t | 1 + gtestpart3_a_idx1 | gtestparted_a_idx_1 | t | 1 +(3 rows) + +INSERT INTO gtestparted(b, c) SELECT g, g + 1 FROM generate_series(16, 1, -1) g; +CLUSTER gtestparted USING gtestparted_a_idx_1; +\d gtestpart2 + Table "generated_virtual_tests.gtestpart2" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+----------------------------- + b | integer | | | + a | integer | | | generated always as (c + 1) + c | integer | | | +Partition of: gtestparted FOR VALUES FROM (10) TO (20) +Indexes: + "gtestpart2_a_idx" btree (a) + "gtestpart2_a_idx_copy" btree (a) CLUSTER + +\d gtestpart3 + Table "generated_virtual_tests.gtestpart3" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+----------------------------- + a | integer | | | generated always as (c + 1) + b | integer | | | + c | integer | | | +Partition of: gtestparted FOR VALUES FROM (1) TO (10) +Indexes: + "gtestpart3_a_idx" btree (a) + "gtestpart3_a_idx1" btree (a) CLUSTER + +--test create table like copy indexes +CREATE TABLE gtestparted_like (LIKE gtestparted INCLUDING ALL); +\d gtestparted_like + Table "generated_virtual_tests.gtestparted_like" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+----------------------------- + b | integer | | | + c | integer | | | + a | integer | | | generated always as (c + 1) +Indexes: + "gtestparted_like_a_idx" btree (a) + "gtestparted_like_a_idx1" btree (a) + +ALTER TABLE gtestparted_like DROP COLUMN a; +\d gtestparted_like + Table "generated_virtual_tests.gtestparted_like" + Column | Type | Collation | Nullable | Default +--------+---------+-----------+----------+--------- + b | integer | | | + c | integer | | | + +CREATE TABLE gtest22c ( + a int, + b int GENERATED ALWAYS AS (a * 2), + c int GENERATED ALWAYS AS (11), + d int GENERATED ALWAYS AS (a * 3), + e int4range GENERATED ALWAYS AS (int4range(a, a + 10)), + e1 int8range GENERATED ALWAYS AS (int8range(a, a + 10)), + f int GENERATED ALWAYS AS (a), + f1 oid GENERATED ALWAYS AS (tableoid) +) WITH (autovacuum_enabled = false); +--index can not based on tableoid column +CREATE INDEX gtest22c_error ON gtest22c (b, f1); +ERROR: index creation on system columns is not supported +LINE 1: CREATE INDEX gtest22c_error ON gtest22c (b, f1); + ^ +CREATE INDEX gtest22c_error ON gtest22c ((f1)); +ERROR: index creation on system columns is not supported +LINE 1: CREATE INDEX gtest22c_error ON gtest22c ((f1)); + ^ +ALTER TABLE gtest22c DROP COLUMN f1; +--index include columns are not supported +-- CREATE INDEX gtest22c_idx1_inc ON gtest22c USING btree(a) include (b,c); +-- CREATE INDEX gtest22c_idx1_inc ON gtest22c USING btree(a) include (f); +--Other index access methods are supported +CREATE INDEX gtest22c_b_idx ON gtest22c USING btree(b, c); +CREATE INDEX gtest22c_d_idx ON gtest22c USING hash(d); +CREATE INDEX gtest22c_e_e1_idx ON gtest22c USING gist(e, e1); +CREATE INDEX gtest22c_e1_idx ON gtest22c USING spgist(e1); --CREATE INDEX gtest22c_expr_idx ON gtest22c ((b * 3)); --CREATE INDEX gtest22c_pred_idx ON gtest22c (a) WHERE b > 0; ---\d gtest22c ---INSERT INTO gtest22c VALUES (1), (2), (3); ---SET enable_seqscan TO off; ---SET enable_bitmapscan TO off; ---EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 4; ---SELECT * FROM gtest22c WHERE b = 4; +\d gtest22c + Table "generated_virtual_tests.gtest22c" + Column | Type | Collation | Nullable | Default +--------+-----------+-----------+----------+-------------------------------------------------------------- + a | integer | | | + b | integer | | | generated always as (a * 2) + c | integer | | | generated always as (11) + d | integer | | | generated always as (a * 3) + e | int4range | | | generated always as (int4range(a, a + 10)) + e1 | int8range | | | generated always as (int8range(a::bigint, (a + 10)::bigint)) + f | integer | | | generated always as (a) +Indexes: + "gtest22c_b_idx" btree (b, c) + "gtest22c_d_idx" hash (d) + "gtest22c_e1_idx" spgist (e1) + "gtest22c_e_e1_idx" gist (e, e1) + +INSERT INTO gtest22c(a) SELECT g FROM generate_series(1, 1_000) g; +SET enable_seqscan TO off; +SET enable_bitmapscan TO off; +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 4; + QUERY PLAN +--------------------------------------------- + Index Scan using gtest22c_b_idx on gtest22c + Index Cond: ((a * 2) = 4) +(2 rows) + +SELECT * FROM gtest22c WHERE b = 4; + a | b | c | d | e | e1 | f +---+---+----+---+--------+--------+--- + 2 | 4 | 11 | 6 | [2,12) | [2,12) | 2 +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 4 AND c = 11; + QUERY PLAN +--------------------------------------------- + Index Scan using gtest22c_b_idx on gtest22c + Index Cond: ((a * 2) = 4) +(2 rows) + +SELECT * FROM gtest22c WHERE b = 4 AND c = 11; + a | b | c | d | e | e1 | f +---+---+----+---+--------+--------+--- + 2 | 4 | 11 | 6 | [2,12) | [2,12) | 2 +(1 row) + +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE d = 6; + QUERY PLAN +--------------------------------------------- + Index Scan using gtest22c_d_idx on gtest22c + Index Cond: ((a * 3) = 6) +(2 rows) + +SELECT * FROM gtest22c WHERE d = 6; + a | b | c | d | e | e1 | f +---+---+----+---+--------+--------+--- + 2 | 4 | 11 | 6 | [2,12) | [2,12) | 2 +(1 row) + +EXPLAIN (COSTS OFF) SELECT count(*) FROM gtest22c WHERE e @> 12 AND e @> 16 AND e1 @> 12::bigint; + QUERY PLAN +---------------------------------------------------------------------------------------------------------------------------------------------------------- + Aggregate + -> Index Scan using gtest22c_e_e1_idx on gtest22c + Index Cond: ((int4range(a, (a + 10)) @> 12) AND (int4range(a, (a + 10)) @> 16) AND (int8range((a)::bigint, ((a + 10))::bigint) @> '12'::bigint)) +(3 rows) + +SELECT count(*) FROM gtest22c WHERE e @> 12 AND e @> 16 AND e1 @> 12::bigint; + count +------- + 6 +(1 row) + +SELECT * FROM gtest22c WHERE e @> 12 AND e @> 16 AND e1 @> 12::bigint; + a | b | c | d | e | e1 | f +----+----+----+----+---------+---------+---- + 7 | 14 | 11 | 21 | [7,17) | [7,17) | 7 + 8 | 16 | 11 | 24 | [8,18) | [8,18) | 8 + 9 | 18 | 11 | 27 | [9,19) | [9,19) | 9 + 10 | 20 | 11 | 30 | [10,20) | [10,20) | 10 + 11 | 22 | 11 | 33 | [11,21) | [11,21) | 11 + 12 | 24 | 11 | 36 | [12,22) | [12,22) | 12 +(6 rows) + +EXPLAIN (COSTS OFF) SELECT count(*) FROM gtest22c WHERE e1 @> 12::bigint AND e1 @> 16::bigint; + QUERY PLAN +----------------------------------------------------------------------------------------------------------------------------------------------------- + Aggregate + -> Index Scan using gtest22c_e1_idx on gtest22c + Index Cond: ((int8range((a)::bigint, ((a + 10))::bigint) @> '12'::bigint) AND (int8range((a)::bigint, ((a + 10))::bigint) @> '16'::bigint)) +(3 rows) + +SELECT count(*) FROM gtest22c WHERE e1 @> 12::bigint AND e1 @> 16::bigint; + count +------- + 6 +(1 row) + +SELECT * FROM gtest22c WHERE e1 @> 12::bigint AND e1 @> 16::bigint; + a | b | c | d | e | e1 | f +----+----+----+----+---------+---------+---- + 12 | 24 | 11 | 36 | [12,22) | [12,22) | 12 + 11 | 22 | 11 | 33 | [11,21) | [11,21) | 11 + 10 | 20 | 11 | 30 | [10,20) | [10,20) | 10 + 9 | 18 | 11 | 27 | [9,19) | [9,19) | 9 + 8 | 16 | 11 | 24 | [8,18) | [8,18) | 8 + 7 | 14 | 11 | 21 | [7,17) | [7,17) | 7 +(6 rows) + +--column drop then the index over that column should also being dropped +ALTER TABLE gtest22c DROP COLUMN e; +CLUSTER gtest22c USING gtest22c_b_idx; +\d gtest22c + Table "generated_virtual_tests.gtest22c" + Column | Type | Collation | Nullable | Default +--------+-----------+-----------+----------+-------------------------------------------------------------- + a | integer | | | + b | integer | | | generated always as (a * 2) + c | integer | | | generated always as (11) + d | integer | | | generated always as (a * 3) + e1 | int8range | | | generated always as (int8range(a::bigint, (a + 10)::bigint)) + f | integer | | | generated always as (a) +Indexes: + "gtest22c_b_idx" btree (b, c) CLUSTER + "gtest22c_d_idx" hash (d) + "gtest22c_e1_idx" spgist (e1) + --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b * 3 = 6; --SELECT * FROM gtest22c WHERE b * 3 = 6; --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE a = 1 AND b > 0; --SELECT * FROM gtest22c WHERE a = 1 AND b > 0; ---ALTER TABLE gtest22c ALTER COLUMN b SET EXPRESSION AS (a * 4); ---ANALYZE gtest22c; ---EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 8; ---SELECT * FROM gtest22c WHERE b = 8; +ALTER TABLE gtest22c ALTER COLUMN b SET EXPRESSION AS (a * 4); +ANALYZE gtest22c; +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 8; + QUERY PLAN +--------------------------------------------- + Index Scan using gtest22c_b_idx on gtest22c + Index Cond: ((a * 4) = 8) +(2 rows) + +SELECT * FROM gtest22c WHERE b = 8; + a | b | c | d | e1 | f +---+---+----+---+--------+--- + 2 | 8 | 11 | 6 | [2,12) | 2 +(1 row) + --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b * 3 = 12; --SELECT * FROM gtest22c WHERE b * 3 = 12; --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE a = 1 AND b > 0; --SELECT * FROM gtest22c WHERE a = 1 AND b > 0; ---RESET enable_seqscan; ---RESET enable_bitmapscan; +--test index over gin and brin index +RESET enable_bitmapscan; +CREATE TABLE t2( + j jsonb, + j1 jsonb GENERATED ALWAYS AS (j || '{"hello": "world"}'), + f1 interval, + f2 interval GENERATED ALWAYS AS ( f1 + interval '1 day')); +INSERT INTO t2(j, f1) SELECT i::text::jsonb, (i || ' days')::interval FROM generate_series(100, 240) s(i); +INSERT INTO t2(f1) VALUES ('-infinity'), ('infinity'); +CREATE INDEX t2_f1_f2_brin_idx ON t2 USING BRIN ( + f1 interval_minmax_multi_ops, + f2 interval_minmax_multi_ops) WITH (pages_per_range = 1); +CREATE INDEX t2_j1_gin_idx ON t2 USING GIN (j1); +EXPLAIN(COSTS OFF) SELECT count(*) FROM t2 WHERE j1 @> '[{"hello":"world"}]'; + QUERY PLAN +------------------------------------------------------------------------------------------------- + Aggregate + -> Bitmap Heap Scan on t2 + Recheck Cond: ((j || '{"hello": "world"}'::jsonb) @> '[{"hello": "world"}]'::jsonb) + -> Bitmap Index Scan on t2_j1_gin_idx + Index Cond: ((j || '{"hello": "world"}'::jsonb) @> '[{"hello": "world"}]'::jsonb) +(5 rows) + +SELECT count(*) FROM t2 WHERE j1 @> '[{"hello":"world"}]'; + count +------- + 141 +(1 row) + +EXPLAIN(COSTS OFF) SELECT * FROM t2 WHERE f1 > '30 years'::interval AND f2 > '30 years'::interval; + QUERY PLAN +--------------------------------------------------------------------------------------------------------------- + Bitmap Heap Scan on t2 + Recheck Cond: ((f1 > '@ 30 years'::interval) AND ((f1 + '@ 1 day'::interval) > '@ 30 years'::interval)) + -> Bitmap Index Scan on t2_f1_f2_brin_idx + Index Cond: ((f1 > '@ 30 years'::interval) AND ((f1 + '@ 1 day'::interval) > '@ 30 years'::interval)) +(4 rows) + +SELECT * FROM t2 WHERE f1 > '30 years'::interval AND f2 > '30 years'::interval; + j | j1 | f1 | f2 +---+----+----------+---------- + | | infinity | infinity +(1 row) + +RESET enable_seqscan; +RESET enable_bitmapscan; -- foreign keys CREATE TABLE gtest23a (x int PRIMARY KEY, y int); --INSERT INTO gtest23a VALUES (1, 11), (2, 22), (3, 33); @@ -1313,6 +1605,7 @@ ALTER TABLE gtest31_1 ALTER COLUMN b SET EXPRESSION AS ('hello2'); DROP STATISTICS gtest31_2_stat; CREATE INDEX gtest31_2_y_idx ON gtest31_2(((y).b)); ALTER TABLE gtest31_1 ALTER COLUMN b SET EXPRESSION AS ('hello3'); +ERROR: cannot alter table "gtest31_1" because column "gtest31_2.y" uses its row type DROP TABLE gtest31_1, gtest31_2; -- Check it for a partitioned table, too CREATE TABLE gtest31_1 (a int, b text GENERATED ALWAYS AS ('hello') VIRTUAL, c text) PARTITION BY LIST (a); @@ -1685,3 +1978,16 @@ select * from gtest33 where b is null; reset constraint_exclusion; drop table gtest33; +-- system catalog sanity check. If the index is based on a virtual generated +-- column, then the corresponding attribute's attgenerated should be 'v'. +select pi.indrelid::regclass, pa.attnum, + pa.attname, + pa.attgenerated +from pg_index pi, unnest(indattrgenerated) sub(a), pg_attribute pa +where 0 <> a +and pa.attrelid = pi.indrelid and pa.attnum = sub.a +and (pa.attgenerated <> 'v' or pi.indnatts <> pi.indnkeyatts); + indrelid | attnum | attname | attgenerated +----------+--------+---------+-------------- +(0 rows) + diff --git a/src/test/regress/expected/indexing.out b/src/test/regress/expected/indexing.out index 4d29fb85293..df4b700a798 100644 --- a/src/test/regress/expected/indexing.out +++ b/src/test/regress/expected/indexing.out @@ -702,6 +702,51 @@ select relname as child, inhparent::regclass as parent, pg_get_indexdef as child idxpart_a_idx | | CREATE INDEX idxpart_a_idx ON ONLY public.idxpart USING btree (a COLLATE "C") (7 rows) +drop table idxpart; +---test collation with virtual generated column +drop table if exists idxpart, idxpart1, idxpart2; +NOTICE: table "idxpart" does not exist, skipping +NOTICE: table "idxpart1" does not exist, skipping +NOTICE: table "idxpart2" does not exist, skipping +create table idxpart (b text, a text generated always as (b), c text) partition by range (c); +create table idxpart1 (b text, a text generated always as (b || 'h'), c text); +create table idxpart2 (like idxpart including generated); +create index idxpart2_a_idx on idxpart2 (a collate "POSIX"); +create index idxpart2_a_idx1 on idxpart2 (a); +create index idxpart2_a_idx2 on idxpart2 (a collate "C"); +alter table idxpart attach partition idxpart1 for values from ('aaa') to ('bbb'); +alter table idxpart attach partition idxpart2 for values from ('bbb') to ('ccc'); +-- error: An index cannot be created when the partitioned table and its +-- partitions have different generation expressions. +create index idxpart_a on idxpart (a collate "C"); +ERROR: cannot create index on partitioned table "idxpart" +DETAIL: The virtual generated column expression of partitioned table "idxpart" does not match that of table "idxpart1" +alter table idxpart1 alter column a set expression as (b collate "C"); +create index idxpart_a on idxpart (a collate "C"); --error +ERROR: cannot create index on partitioned table "idxpart" +DETAIL: The virtual generated column expression of partitioned table "idxpart" does not match that of table "idxpart1" +alter table idxpart1 alter column a set expression as (b); +create index idxpart_a on idxpart (a collate "C"); ---ok +create table idxpart4 partition of idxpart for values from ('ddd') to ('eee'); +select relname as child, inhparent::regclass as parent, + pi.indkey, pi.indattrgenerated, + (pi.indcollation[0])::regcollation, + pg_get_indexdef as childdef +from pg_class join pg_index pi on pi.indexrelid = pg_class.oid +left join pg_inherits on inhrelid = oid, +lateral pg_get_indexdef(pg_class.oid) +where relkind in ('i', 'I') and relname like 'idxpart%' +order by relname; + child | parent | indkey | indattrgenerated | indcollation | childdef +-----------------+-----------+--------+------------------+--------------+-------------------------------------------------------------------------------- + idxpart1_a_idx | idxpart_a | 1 | 2 | "C" | CREATE INDEX idxpart1_a_idx ON public.idxpart1 USING btree (a COLLATE "C") + idxpart2_a_idx | | 1 | 2 | "POSIX" | CREATE INDEX idxpart2_a_idx ON public.idxpart2 USING btree (a COLLATE "POSIX") + idxpart2_a_idx1 | | 1 | 2 | "default" | CREATE INDEX idxpart2_a_idx1 ON public.idxpart2 USING btree (a) + idxpart2_a_idx2 | idxpart_a | 1 | 2 | "C" | CREATE INDEX idxpart2_a_idx2 ON public.idxpart2 USING btree (a COLLATE "C") + idxpart4_a_idx | idxpart_a | 1 | 2 | "C" | CREATE INDEX idxpart4_a_idx ON public.idxpart4 USING btree (a COLLATE "C") + idxpart_a | | 1 | 2 | "C" | CREATE INDEX idxpart_a ON ONLY public.idxpart USING btree (a COLLATE "C") +(6 rows) + drop table idxpart; -- Verify behavior for opclass (mis)matches create table idxpart (a text) partition by range (a); diff --git a/src/test/regress/sql/alter_table.sql b/src/test/regress/sql/alter_table.sql index 417202430a5..0b215d4daf6 100644 --- a/src/test/regress/sql/alter_table.sql +++ b/src/test/regress/sql/alter_table.sql @@ -156,6 +156,15 @@ ALTER INDEX attmp_idx ALTER COLUMN 4 SET STATISTICS 1000; ALTER INDEX attmp_idx ALTER COLUMN 2 SET STATISTICS -1; +ALTER TABLE attmp + ADD COLUMN col1 int GENERATED ALWAYS AS (a), + ADD COLUMN col2 int GENERATED ALWAYS AS (a + 1); + +CREATE INDEX attmp_idx1 ON attmp (a, col1, col2); +ALTER INDEX attmp_idx1 ALTER COLUMN 1 SET STATISTICS 1000; +ALTER INDEX attmp_idx1 ALTER COLUMN 2 SET STATISTICS 1000; +ALTER INDEX attmp_idx1 ALTER COLUMN 3 SET STATISTICS 1000; +\d+ attmp_idx1 DROP TABLE attmp; diff --git a/src/test/regress/sql/collate.icu.utf8.sql b/src/test/regress/sql/collate.icu.utf8.sql index b6c54503d21..ab70aaed7e2 100644 --- a/src/test/regress/sql/collate.icu.utf8.sql +++ b/src/test/regress/sql/collate.icu.utf8.sql @@ -1000,6 +1000,14 @@ INSERT INTO t5 (a, b) values (1, 'D1'), (2, 'D2'), (3, 'd1'); -- rewriting.) SELECT * FROM t5 ORDER BY c ASC, a ASC; +CREATE INDEX t5_idx1 ON t5 USING btree((b COLLATE "POSIX")); +CREATE INDEX t5_idx2 ON t5 USING btree((b)); +SELECT indexrelid::regclass, indrelid::regclass, indnatts, indattrgenerated, + indcollation[0]::regcollation +FROM pg_index +WHERE indrelid = 't5'::regclass +ORDER BY indexrelid; + -- Check that DEFAULT expressions in SQL/JSON functions use the same collation -- as the RETURNING type. Mismatched collations should raise an error. CREATE DOMAIN d1 AS text COLLATE case_insensitive; diff --git a/src/test/regress/sql/fast_default.sql b/src/test/regress/sql/fast_default.sql index 068dd0bc8aa..b39e76bcfc3 100644 --- a/src/test/regress/sql/fast_default.sql +++ b/src/test/regress/sql/fast_default.sql @@ -77,6 +77,12 @@ ALTER TABLE has_volatile ALTER COLUMN col1 SET DATA TYPE float8, -- stored generated columns need a rewrite ALTER TABLE has_volatile ADD col7 int GENERATED ALWAYS AS (55) stored; +-- if there is any index over virtual generated columns, +-- change generation expression need rewrite +CREATE INDEX on has_volatile(col6); +ALTER TABLE has_volatile ALTER COLUMN col6 SET EXPRESSION AS (col1 * 3); +-- table rewrite again. +ALTER TABLE has_volatile ALTER COLUMN col6 SET DATA TYPE INT8; -- Test a large sample of different datatypes diff --git a/src/test/regress/sql/generated_virtual.sql b/src/test/regress/sql/generated_virtual.sql index 81152b39a79..c8e933c8300 100644 --- a/src/test/regress/sql/generated_virtual.sql +++ b/src/test/regress/sql/generated_virtual.sql @@ -1,6 +1,4 @@ -- keep these tests aligned with generated_stored.sql - - CREATE SCHEMA generated_virtual_tests; GRANT USAGE ON SCHEMA generated_virtual_tests TO PUBLIC; SET search_path = generated_virtual_tests; @@ -396,32 +394,138 @@ CREATE TABLE gtest22b (a int, b int GENERATED ALWAYS AS (a / 2) VIRTUAL, PRIMARY --INSERT INTO gtest22b VALUES (2); -- indexes -CREATE TABLE gtest22c (a int, b int GENERATED ALWAYS AS (a * 2) VIRTUAL); ---CREATE INDEX gtest22c_b_idx ON gtest22c (b); +CREATE TABLE gtestparted (b integer, c integer, a integer GENERATED ALWAYS AS (c+1)) PARTITION BY RANGE (b); +CREATE TABLE gtestpart3 (a integer GENERATED ALWAYS AS (c), b integer, c integer); + +ALTER TABLE gtestparted ATTACH PARTITION gtestpart3 FOR VALUES FROM (1) to (10); +-- error: An index cannot be created when the partitioned table and its +-- partitions have different generation expressions. +CREATE INDEX gtestparted_a_idx_error on gtestparted(a); +ALTER TABLE gtestpart3 ALTER COLUMN a SET EXPRESSION AS (c + 1); + +CREATE TABLE gtestpart1 (c integer, a integer GENERATED ALWAYS AS (c), b integer); +CREATE TABLE gtestpart2 (b integer, a integer GENERATED ALWAYS AS (c+1), c integer); +CREATE INDEX gtestpart1_a_idx on gtestpart1(a); +CREATE INDEX gtestpart2_a_idx on gtestpart2(a); +CREATE INDEX gtestpart2_a_idx_copy on gtestpart2(a); +CREATE INDEX gtestparted_a_idx on gtestparted(a); + +-- error: An index cannot be created when the partitioned table and its +-- partitions have different generation expressions. +ALTER TABLE gtestparted ATTACH PARTITION gtestpart1 FOR VALUES FROM (10) TO (20); + +ALTER TABLE gtestparted ATTACH PARTITION gtestpart2 FOR VALUES FROM (10) TO (20); --ok +SELECT * FROM pg_partition_tree('gtestparted_a_idx'::regclass); + +ALTER INDEX gtestparted_a_idx ATTACH PARTITION gtestpart2_a_idx_copy; --error +ALTER INDEX gtestparted_a_idx ATTACH PARTITION gtestpart2_a_idx; --ok + +CREATE INDEX gtestparted_a_idx_1 on gtestparted(a); +--now index gtestpart2_a_idx_copy should attach to the partition tree +SELECT * FROM pg_partition_tree('gtestparted_a_idx_1'::regclass); + +INSERT INTO gtestparted(b, c) SELECT g, g + 1 FROM generate_series(16, 1, -1) g; +CLUSTER gtestparted USING gtestparted_a_idx_1; +\d gtestpart2 +\d gtestpart3 + +--test create table like copy indexes +CREATE TABLE gtestparted_like (LIKE gtestparted INCLUDING ALL); +\d gtestparted_like +ALTER TABLE gtestparted_like DROP COLUMN a; +\d gtestparted_like + +CREATE TABLE gtest22c ( + a int, + b int GENERATED ALWAYS AS (a * 2), + c int GENERATED ALWAYS AS (11), + d int GENERATED ALWAYS AS (a * 3), + e int4range GENERATED ALWAYS AS (int4range(a, a + 10)), + e1 int8range GENERATED ALWAYS AS (int8range(a, a + 10)), + f int GENERATED ALWAYS AS (a), + f1 oid GENERATED ALWAYS AS (tableoid) +) WITH (autovacuum_enabled = false); + +--index can not based on tableoid column +CREATE INDEX gtest22c_error ON gtest22c (b, f1); +CREATE INDEX gtest22c_error ON gtest22c ((f1)); +ALTER TABLE gtest22c DROP COLUMN f1; + +--index include columns are not supported +-- CREATE INDEX gtest22c_idx1_inc ON gtest22c USING btree(a) include (b,c); +-- CREATE INDEX gtest22c_idx1_inc ON gtest22c USING btree(a) include (f); + +--Other index access methods are supported +CREATE INDEX gtest22c_b_idx ON gtest22c USING btree(b, c); +CREATE INDEX gtest22c_d_idx ON gtest22c USING hash(d); +CREATE INDEX gtest22c_e_e1_idx ON gtest22c USING gist(e, e1); +CREATE INDEX gtest22c_e1_idx ON gtest22c USING spgist(e1); + --CREATE INDEX gtest22c_expr_idx ON gtest22c ((b * 3)); --CREATE INDEX gtest22c_pred_idx ON gtest22c (a) WHERE b > 0; ---\d gtest22c +\d gtest22c + +INSERT INTO gtest22c(a) SELECT g FROM generate_series(1, 1_000) g; +SET enable_seqscan TO off; +SET enable_bitmapscan TO off; + +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 4; +SELECT * FROM gtest22c WHERE b = 4; +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 4 AND c = 11; +SELECT * FROM gtest22c WHERE b = 4 AND c = 11; + +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE d = 6; +SELECT * FROM gtest22c WHERE d = 6; + +EXPLAIN (COSTS OFF) SELECT count(*) FROM gtest22c WHERE e @> 12 AND e @> 16 AND e1 @> 12::bigint; +SELECT count(*) FROM gtest22c WHERE e @> 12 AND e @> 16 AND e1 @> 12::bigint; +SELECT * FROM gtest22c WHERE e @> 12 AND e @> 16 AND e1 @> 12::bigint; + +EXPLAIN (COSTS OFF) SELECT count(*) FROM gtest22c WHERE e1 @> 12::bigint AND e1 @> 16::bigint; +SELECT count(*) FROM gtest22c WHERE e1 @> 12::bigint AND e1 @> 16::bigint; +SELECT * FROM gtest22c WHERE e1 @> 12::bigint AND e1 @> 16::bigint; + +--column drop then the index over that column should also being dropped +ALTER TABLE gtest22c DROP COLUMN e; +CLUSTER gtest22c USING gtest22c_b_idx; +\d gtest22c ---INSERT INTO gtest22c VALUES (1), (2), (3); ---SET enable_seqscan TO off; ---SET enable_bitmapscan TO off; ---EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 4; ---SELECT * FROM gtest22c WHERE b = 4; --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b * 3 = 6; --SELECT * FROM gtest22c WHERE b * 3 = 6; --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE a = 1 AND b > 0; --SELECT * FROM gtest22c WHERE a = 1 AND b > 0; ---ALTER TABLE gtest22c ALTER COLUMN b SET EXPRESSION AS (a * 4); ---ANALYZE gtest22c; ---EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 8; ---SELECT * FROM gtest22c WHERE b = 8; +ALTER TABLE gtest22c ALTER COLUMN b SET EXPRESSION AS (a * 4); +ANALYZE gtest22c; +EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b = 8; +SELECT * FROM gtest22c WHERE b = 8; --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE b * 3 = 12; --SELECT * FROM gtest22c WHERE b * 3 = 12; --EXPLAIN (COSTS OFF) SELECT * FROM gtest22c WHERE a = 1 AND b > 0; --SELECT * FROM gtest22c WHERE a = 1 AND b > 0; ---RESET enable_seqscan; ---RESET enable_bitmapscan; + +--test index over gin and brin index +RESET enable_bitmapscan; +CREATE TABLE t2( + j jsonb, + j1 jsonb GENERATED ALWAYS AS (j || '{"hello": "world"}'), + f1 interval, + f2 interval GENERATED ALWAYS AS ( f1 + interval '1 day')); +INSERT INTO t2(j, f1) SELECT i::text::jsonb, (i || ' days')::interval FROM generate_series(100, 240) s(i); +INSERT INTO t2(f1) VALUES ('-infinity'), ('infinity'); +CREATE INDEX t2_f1_f2_brin_idx ON t2 USING BRIN ( + f1 interval_minmax_multi_ops, + f2 interval_minmax_multi_ops) WITH (pages_per_range = 1); +CREATE INDEX t2_j1_gin_idx ON t2 USING GIN (j1); + +EXPLAIN(COSTS OFF) SELECT count(*) FROM t2 WHERE j1 @> '[{"hello":"world"}]'; +SELECT count(*) FROM t2 WHERE j1 @> '[{"hello":"world"}]'; + +EXPLAIN(COSTS OFF) SELECT * FROM t2 WHERE f1 > '30 years'::interval AND f2 > '30 years'::interval; +SELECT * FROM t2 WHERE f1 > '30 years'::interval AND f2 > '30 years'::interval; + +RESET enable_seqscan; +RESET enable_bitmapscan; -- foreign keys CREATE TABLE gtest23a (x int PRIMARY KEY, y int); @@ -891,3 +995,13 @@ select * from gtest33 where b is null; reset constraint_exclusion; drop table gtest33; + +-- system catalog sanity check. If the index is based on a virtual generated +-- column, then the corresponding attribute's attgenerated should be 'v'. +select pi.indrelid::regclass, pa.attnum, + pa.attname, + pa.attgenerated +from pg_index pi, unnest(indattrgenerated) sub(a), pg_attribute pa +where 0 <> a +and pa.attrelid = pi.indrelid and pa.attnum = sub.a +and (pa.attgenerated <> 'v' or pi.indnatts <> pi.indnkeyatts); diff --git a/src/test/regress/sql/indexing.sql b/src/test/regress/sql/indexing.sql index b5cb01c2d70..192d67be1ec 100644 --- a/src/test/regress/sql/indexing.sql +++ b/src/test/regress/sql/indexing.sql @@ -328,6 +328,36 @@ select relname as child, inhparent::regclass as parent, pg_get_indexdef as child where relkind in ('i', 'I') and relname like 'idxpart%' order by relname; drop table idxpart; +---test collation with virtual generated column +drop table if exists idxpart, idxpart1, idxpart2; +create table idxpart (b text, a text generated always as (b), c text) partition by range (c); +create table idxpart1 (b text, a text generated always as (b || 'h'), c text); +create table idxpart2 (like idxpart including generated); +create index idxpart2_a_idx on idxpart2 (a collate "POSIX"); +create index idxpart2_a_idx1 on idxpart2 (a); +create index idxpart2_a_idx2 on idxpart2 (a collate "C"); +alter table idxpart attach partition idxpart1 for values from ('aaa') to ('bbb'); +alter table idxpart attach partition idxpart2 for values from ('bbb') to ('ccc'); +-- error: An index cannot be created when the partitioned table and its +-- partitions have different generation expressions. +create index idxpart_a on idxpart (a collate "C"); +alter table idxpart1 alter column a set expression as (b collate "C"); +create index idxpart_a on idxpart (a collate "C"); --error +alter table idxpart1 alter column a set expression as (b); +create index idxpart_a on idxpart (a collate "C"); ---ok +create table idxpart4 partition of idxpart for values from ('ddd') to ('eee'); +select relname as child, inhparent::regclass as parent, + pi.indkey, pi.indattrgenerated, + (pi.indcollation[0])::regcollation, + pg_get_indexdef as childdef +from pg_class join pg_index pi on pi.indexrelid = pg_class.oid +left join pg_inherits on inhrelid = oid, +lateral pg_get_indexdef(pg_class.oid) +where relkind in ('i', 'I') and relname like 'idxpart%' +order by relname; + +drop table idxpart; + -- Verify behavior for opclass (mis)matches create table idxpart (a text) partition by range (a); create table idxpart1 (like idxpart); -- 2.34.1
