The previous discussion of automatic partition creation [1] has
addressed static and dynamic creation of partitions and ended up with
several syntax proposals.
In this thread, I want to continue this work.
Attached is PoC for static partition creation. The patch core is quite
straightforward. It adds one more transform clause to convert given
partitioning specification into several CREATE TABLE statements.
The patch implements following syntax:
CREATE TABLE ... PARTITION BY partition_method (list_of_columns)
partition_auto_create_clause
where partition_auto_create_clause is
CONFIGURATION [IMMEDIATE| DEFERRED] USING partition_bound_spec
and partition_bound_spec is:
MODULUS integer | VALUES IN (expr [,...]) [, ....] | INTERVAL
range_step FROM range_start TO range_end
For more examples check auto_partitions.sql in the patch.
TODO:
- CONFIGURATION is just an existing keyword, that I picked as a stub.
Ideas on better wording are welcome.
- IMMEDIATE| DEFERRED is optional, DEFERRED is not implemented yet
I wonder, is it worth placing a stub for dynamic partitioning, or we can
rather add these keywords later.
- HASH and LIST static partitioning works as expected.
Testing and feedback are welcome.
- RANGE partitioning is not really implemented in this patch.
Now it only accepts interval data type as 'interval' and respectively
date types as range_start and range_end expressions.
Only one partition is created. I found it difficult to implement the
generation of bounds using internal functions and data types.
Both existing solutions (pg_pathman and pg_partman) rely on SQL level
routines [2].
I am going to implement this via SPI, which allow to simplify checks and
calculations. Do you see any pitfalls in this approach?
- Partition naming. Now partition names for all methods look like
$tablename_$partnum
Do we want more intelligence here? Now we have
RunObjectPostCreateHook(), which allows to rename the table.
To make it more user-friendly, we can later implement pl/pgsql function
that sets the callback, as it is done in pg_pathman set_init_callback() [3].
- Current design doesn't allow to create default partition
automatically. Do we need this functionality?
- Do you see any restrictions for future extensibility (dynamic
partitioning, init_callback, etc.) in the proposed design ?
I expect this to be a long discussion, so here is the wiki page [4] to
fix important questions and final agreements.
[1]
https://www.postgresql.org/message-id/flat/alpine.DEB.2.21.1907150711080.22273%40lancre
[2]
https://github.com/postgrespro/pg_pathman/blob/dbcbd02e411e6acea6d97f572234746007979538/range.sql#L99
[3] https://github.com/postgrespro/pg_pathman#additional-parameters
[4] https://wiki.postgresql.org/wiki/Declarative_partitioning_improvements
--
Anastasia Lubennikova
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
>From 5808c5e843cac1e1383366a5cbff116eaa433f90 Mon Sep 17 00:00:00 2001
From: anastasia <a.lubennik...@postgrespro.ru>
Date: Fri, 3 Jul 2020 03:34:24 +0300
Subject: [PATCH] WIP create partitions automatically Implement new syntax to
generate bounds for HASH, LIST and RANGE partitions. Implement automatic
partition creation for HASH and LIST. Check new regression test
'auto_partitions.sql' for syntax examples
---
src/backend/commands/tablecmds.c | 7 +
src/backend/nodes/copyfuncs.c | 33 ++++
src/backend/nodes/equalfuncs.c | 29 +++
src/backend/nodes/outfuncs.c | 28 +++
src/backend/nodes/readfuncs.c | 33 ++++
src/backend/parser/gram.y | 160 ++++++++++++++---
src/backend/parser/parse_utilcmd.c | 166 ++++++++++++++++++
src/include/nodes/nodes.h | 2 +
src/include/nodes/parsenodes.h | 43 +++++
src/include/parser/kwlist.h | 1 +
src/include/partitioning/partdefs.h | 4 +
src/test/regress/expected/auto_partitions.out | 0
src/test/regress/parallel_schedule | 2 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/auto_partitions.sql | 43 +++++
15 files changed, 523 insertions(+), 29 deletions(-)
create mode 100644 src/test/regress/expected/auto_partitions.out
create mode 100644 src/test/regress/sql/auto_partitions.sql
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f79044f39f..7b2c651952 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -628,6 +628,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
else
partitioned = false;
+ if (!partitioned && stmt->partautocreate)
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_TABLE_DEFINITION),
+ errmsg("PARTITION bounds can only be used on partitioned tables")));
+ }
+
/*
* Look up the namespace in which we are supposed to create the relation,
* check we have permission to create there, lock it against concurrent
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index d8cf87e6d0..74a305c5d1 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -3393,6 +3393,7 @@ CopyCreateStmtFields(const CreateStmt *from, CreateStmt *newnode)
COPY_NODE_FIELD(inhRelations);
COPY_NODE_FIELD(partspec);
COPY_NODE_FIELD(partbound);
+ COPY_NODE_FIELD(partautocreate);
COPY_NODE_FIELD(ofTypename);
COPY_NODE_FIELD(constraints);
COPY_NODE_FIELD(options);
@@ -4651,6 +4652,32 @@ _copyPartitionBoundSpec(const PartitionBoundSpec *from)
return newnode;
}
+static PartitionBoundAutoSpec *
+_copyPartitionBoundAutoSpec(const PartitionBoundAutoSpec *from)
+{
+ PartitionBoundAutoSpec *newnode = makeNode(PartitionBoundAutoSpec);
+
+ COPY_SCALAR_FIELD(strategy);
+ COPY_SCALAR_FIELD(modulus);
+ COPY_NODE_FIELD(listdatumsList);
+ COPY_NODE_FIELD(interval);
+ COPY_NODE_FIELD(lowerdatums);
+ COPY_NODE_FIELD(upperdatums);
+
+ return newnode;
+}
+
+static PartitionAutoCreate *
+_copyPartitionAutoCreate(const PartitionAutoCreate *from)
+{
+ PartitionAutoCreate *newnode = makeNode(PartitionAutoCreate);
+
+ COPY_SCALAR_FIELD(is_deferred);
+ COPY_NODE_FIELD(bound);
+
+ return newnode;
+}
+
static PartitionRangeDatum *
_copyPartitionRangeDatum(const PartitionRangeDatum *from)
{
@@ -5700,6 +5727,12 @@ copyObjectImpl(const void *from)
case T_PartitionBoundSpec:
retval = _copyPartitionBoundSpec(from);
break;
+ case T_PartitionBoundAutoSpec:
+ retval = _copyPartitionBoundAutoSpec(from);
+ break;
+ case T_PartitionAutoCreate:
+ retval = _copyPartitionAutoCreate(from);
+ break;
case T_PartitionRangeDatum:
retval = _copyPartitionRangeDatum(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 627b026b19..5ca9ed6a90 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2917,6 +2917,29 @@ _equalPartitionBoundSpec(const PartitionBoundSpec *a, const PartitionBoundSpec *
return true;
}
+static bool
+_equalPartitionBoundAutoSpec(const PartitionBoundAutoSpec *a,
+ const PartitionBoundAutoSpec *b)
+{
+ COMPARE_SCALAR_FIELD(strategy);
+ COMPARE_SCALAR_FIELD(modulus);
+ COMPARE_NODE_FIELD(listdatumsList);
+ COMPARE_NODE_FIELD(interval);
+ COMPARE_NODE_FIELD(lowerdatums);
+ COMPARE_NODE_FIELD(upperdatums);
+
+ return true;
+}
+
+static bool
+_equalPartitionAutoCreate(const PartitionAutoCreate *a, const PartitionAutoCreate *b)
+{
+ COMPARE_SCALAR_FIELD(is_deferred);
+ COMPARE_NODE_FIELD(bound);
+
+ return true;
+}
+
static bool
_equalPartitionRangeDatum(const PartitionRangeDatum *a, const PartitionRangeDatum *b)
{
@@ -3752,6 +3775,12 @@ equal(const void *a, const void *b)
case T_PartitionBoundSpec:
retval = _equalPartitionBoundSpec(a, b);
break;
+ case T_PartitionBoundAutoSpec:
+ retval = _equalPartitionBoundAutoSpec(a, b);
+ break;
+ case T_PartitionAutoCreate:
+ retval = _equalPartitionAutoCreate(a, b);
+ break;
case T_PartitionRangeDatum:
retval = _equalPartitionRangeDatum(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515d..35b1438197 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -3661,6 +3661,28 @@ _outPartitionBoundSpec(StringInfo str, const PartitionBoundSpec *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outPartitionBoundAutoSpec(StringInfo str, const PartitionBoundAutoSpec *node)
+{
+ WRITE_NODE_TYPE("PARTITIONBOUNDAUTOSPEC");
+
+ WRITE_CHAR_FIELD(strategy);
+ WRITE_INT_FIELD(modulus);
+ WRITE_NODE_FIELD(listdatumsList);
+ WRITE_NODE_FIELD(interval);
+ WRITE_NODE_FIELD(lowerdatums);
+ WRITE_NODE_FIELD(upperdatums);
+}
+
+static void
+_outPartitionAutoCreate(StringInfo str, const PartitionAutoCreate *node)
+{
+ WRITE_NODE_TYPE("PartitionAutoCreate");
+
+ WRITE_BOOL_FIELD(is_deferred);
+ WRITE_NODE_FIELD(bound);
+}
+
static void
_outPartitionRangeDatum(StringInfo str, const PartitionRangeDatum *node)
{
@@ -4334,6 +4356,12 @@ outNode(StringInfo str, const void *obj)
case T_PartitionBoundSpec:
_outPartitionBoundSpec(str, obj);
break;
+ case T_PartitionBoundAutoSpec:
+ _outPartitionBoundAutoSpec(str, obj);
+ break;
+ case T_PartitionAutoCreate:
+ _outPartitionAutoCreate(str, obj);
+ break;
case T_PartitionRangeDatum:
_outPartitionRangeDatum(str, obj);
break;
diff --git a/src/backend/nodes/readfuncs.c b/src/backend/nodes/readfuncs.c
index 42050ab719..fa2b08316b 100644
--- a/src/backend/nodes/readfuncs.c
+++ b/src/backend/nodes/readfuncs.c
@@ -2602,6 +2602,35 @@ _readPartitionBoundSpec(void)
READ_DONE();
}
+static PartitionBoundAutoSpec *
+_readPartitionBoundAutoSpec(void)
+{
+ READ_LOCALS(PartitionBoundAutoSpec);
+
+ READ_CHAR_FIELD(strategy);
+ READ_INT_FIELD(modulus);
+ READ_NODE_FIELD(listdatumsList);
+ READ_NODE_FIELD(interval);
+ READ_NODE_FIELD(lowerdatums);
+ READ_NODE_FIELD(upperdatums);
+
+ READ_DONE();
+}
+
+/*
+ * _readPartitionAutoCreate
+ */
+static PartitionAutoCreate *
+_readPartitionAutoCreate(void)
+{
+ READ_LOCALS(PartitionAutoCreate);
+
+ READ_BOOL_FIELD(is_deferred);
+ READ_NODE_FIELD(bound);
+
+ READ_DONE();
+}
+
/*
* _readPartitionRangeDatum
*/
@@ -2880,6 +2909,10 @@ parseNodeString(void)
return_value = _readPartitionBoundSpec();
else if (MATCH("PARTITIONRANGEDATUM", 19))
return_value = _readPartitionRangeDatum();
+ else if (MATCH("PARTITIONAUTPBOUNDSPEC", 21))
+ return_value = _readPartitionAutoCreate();
+ else if (MATCH("PARTITIONBOUNDAUTOSPEC", 22))
+ return_value = _readPartitionBoundAutoSpec();
else
{
elog(ERROR, "badly formatted node string \"%.32s\"...", token);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4ff35095b8..3c62837763 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -249,6 +249,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
PartitionElem *partelem;
PartitionSpec *partspec;
PartitionBoundSpec *partboundspec;
+ PartitionBoundAutoSpec *partboundautospec;
+ PartitionAutoCreate *partautocreate;
RoleSpec *rolespec;
struct SelectLimit *selectlimit;
}
@@ -599,6 +601,11 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <partboundautospec> PartitionBoundAutoSpec values_in_clause
+%type <partautocreate> PartitionAutoCreate OptPartitionAutoCreate
+%type <boolean> opt_part_deferred
+%type <node> range_interval
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -669,7 +676,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
LEADING LEAKPROOF LEAST LEFT LEVEL LIKE LIMIT LISTEN LOAD LOCAL
LOCALTIME LOCALTIMESTAMP LOCATION LOCK_P LOCKED LOGGED
- MAPPING MATCH MATERIALIZED MAXVALUE METHOD MINUTE_P MINVALUE MODE MONTH_P MOVE
+ MAPPING MATCH MATERIALIZED MAXVALUE METHOD MINUTE_P MINVALUE MODE MODULUS MONTH_P MOVE
NAME_P NAMES NATIONAL NATURAL NCHAR NEW NEXT NFC NFD NFKC NFKD NO NONE
NORMALIZE NORMALIZED
@@ -3179,7 +3186,8 @@ copy_generic_opt_arg_list_item:
*****************************************************************************/
CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
- OptInherit OptPartitionSpec table_access_method_clause OptWith
+ OptInherit OptPartitionSpec OptPartitionAutoCreate
+ table_access_method_clause OptWith
OnCommitOption OptTableSpace
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3188,17 +3196,19 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->tableElts = $6;
n->inhRelations = $8;
n->partspec = $9;
+ n->partautocreate = $10;
n->ofTypename = NULL;
n->constraints = NIL;
- n->accessMethod = $10;
- n->options = $11;
- n->oncommit = $12;
- n->tablespacename = $13;
+ n->accessMethod = $11;
+ n->options = $12;
+ n->oncommit = $13;
+ n->tablespacename = $14;
n->if_not_exists = false;
$$ = (Node *)n;
}
| CREATE OptTemp TABLE IF_P NOT EXISTS qualified_name '('
- OptTableElementList ')' OptInherit OptPartitionSpec table_access_method_clause
+ OptTableElementList ')' OptInherit OptPartitionSpec
+ OptPartitionAutoCreate table_access_method_clause
OptWith OnCommitOption OptTableSpace
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3207,17 +3217,19 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->tableElts = $9;
n->inhRelations = $11;
n->partspec = $12;
+ n->partautocreate = $13;
n->ofTypename = NULL;
n->constraints = NIL;
- n->accessMethod = $13;
- n->options = $14;
- n->oncommit = $15;
- n->tablespacename = $16;
+ n->accessMethod = $14;
+ n->options = $15;
+ n->oncommit = $16;
+ n->tablespacename = $17;
n->if_not_exists = true;
$$ = (Node *)n;
}
| CREATE OptTemp TABLE qualified_name OF any_name
- OptTypedTableElementList OptPartitionSpec table_access_method_clause
+ OptTypedTableElementList OptPartitionSpec
+ OptPartitionAutoCreate table_access_method_clause
OptWith OnCommitOption OptTableSpace
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3226,18 +3238,20 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->tableElts = $7;
n->inhRelations = NIL;
n->partspec = $8;
+ n->partautocreate = $9;
n->ofTypename = makeTypeNameFromNameList($6);
n->ofTypename->location = @6;
n->constraints = NIL;
- n->accessMethod = $9;
- n->options = $10;
- n->oncommit = $11;
- n->tablespacename = $12;
+ n->accessMethod = $10;
+ n->options = $11;
+ n->oncommit = $12;
+ n->tablespacename = $13;
n->if_not_exists = false;
$$ = (Node *)n;
}
| CREATE OptTemp TABLE IF_P NOT EXISTS qualified_name OF any_name
- OptTypedTableElementList OptPartitionSpec table_access_method_clause
+ OptTypedTableElementList OptPartitionSpec
+ OptPartitionAutoCreate table_access_method_clause
OptWith OnCommitOption OptTableSpace
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3246,18 +3260,20 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->tableElts = $10;
n->inhRelations = NIL;
n->partspec = $11;
+ n->partautocreate = $12;
n->ofTypename = makeTypeNameFromNameList($9);
n->ofTypename->location = @9;
n->constraints = NIL;
- n->accessMethod = $12;
- n->options = $13;
- n->oncommit = $14;
- n->tablespacename = $15;
+ n->accessMethod = $13;
+ n->options = $14;
+ n->oncommit = $15;
+ n->tablespacename = $16;
n->if_not_exists = true;
$$ = (Node *)n;
}
| CREATE OptTemp TABLE qualified_name PARTITION OF qualified_name
OptTypedTableElementList PartitionBoundSpec OptPartitionSpec
+ OptPartitionAutoCreate
table_access_method_clause OptWith OnCommitOption OptTableSpace
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3267,17 +3283,19 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->inhRelations = list_make1($7);
n->partbound = $9;
n->partspec = $10;
+ n->partautocreate = $11;
n->ofTypename = NULL;
n->constraints = NIL;
- n->accessMethod = $11;
- n->options = $12;
- n->oncommit = $13;
- n->tablespacename = $14;
+ n->accessMethod = $12;
+ n->options = $13;
+ n->oncommit = $14;
+ n->tablespacename = $15;
n->if_not_exists = false;
$$ = (Node *)n;
}
| CREATE OptTemp TABLE IF_P NOT EXISTS qualified_name PARTITION OF
qualified_name OptTypedTableElementList PartitionBoundSpec OptPartitionSpec
+ OptPartitionAutoCreate
table_access_method_clause OptWith OnCommitOption OptTableSpace
{
CreateStmt *n = makeNode(CreateStmt);
@@ -3287,12 +3305,13 @@ CreateStmt: CREATE OptTemp TABLE qualified_name '(' OptTableElementList ')'
n->inhRelations = list_make1($10);
n->partbound = $12;
n->partspec = $13;
+ n->partautocreate = $14;
n->ofTypename = NULL;
n->constraints = NIL;
- n->accessMethod = $14;
- n->options = $15;
- n->oncommit = $16;
- n->tablespacename = $17;
+ n->accessMethod = $15;
+ n->options = $16;
+ n->oncommit = $17;
+ n->tablespacename = $18;
n->if_not_exists = true;
$$ = (Node *)n;
}
@@ -3958,6 +3977,88 @@ part_elem: ColId opt_collate opt_class
}
;
+/* Optional partition automatic creation specification */
+OptPartitionAutoCreate: PartitionAutoCreate { $$ = $1; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+/* XXX
+ * CONFIGURATION is just a random keyword that exists already and fits here.
+ * Any ideas on better wording?
+ */
+PartitionAutoCreate: CONFIGURATION opt_part_deferred USING PartitionBoundAutoSpec
+ {
+ PartitionAutoCreate *n = makeNode(PartitionAutoCreate);
+
+ n->is_deferred = $2;
+ n->bound = $4;
+
+ $$ = n;
+ }
+ ;
+
+opt_part_deferred:
+ DEFERRED { $$ = true; }
+ | IMMEDIATE { $$ = false; }
+ | /* EMPTY*/ { $$ = false;}
+ ;
+
+PartitionBoundAutoSpec:
+ /* a HASH partition */
+ MODULUS Iconst
+ {
+ PartitionBoundAutoSpec *n = makeNode(PartitionBoundAutoSpec);
+
+ n->strategy = PARTITION_STRATEGY_HASH;
+ n->modulus = (int16) $2;
+
+ $$ = n;
+ }
+
+ /* a LIST partition */
+ | values_in_clause { $$ = $1; }
+
+ /* a RANGE partition */
+ | range_interval FROM '(' expr_list ')' TO '(' expr_list ')'
+ {
+ PartitionBoundAutoSpec *n = makeNode(PartitionBoundAutoSpec);
+
+ n->strategy = PARTITION_STRATEGY_RANGE;
+ n->interval = $1;
+ n->lowerdatums = $4;
+ n->upperdatums = $8;
+
+ $$ = n;
+ }
+ ;
+
+/* TODO allow not only interval, but also other types of Const values */
+range_interval:
+ ConstInterval Sconst opt_interval
+ {
+ TypeName *t = $1;
+ t->typmods = $3;
+ $$ = makeStringConstCast($2, @2, t);
+ }
+ ;
+
+values_in_clause:
+ VALUES IN_P '(' expr_list ')'
+ {
+ PartitionBoundAutoSpec *n = makeNode(PartitionBoundAutoSpec);
+ n->strategy = PARTITION_STRATEGY_LIST;
+ n->listdatumsList = list_make1($4);
+ $$ = (PartitionBoundAutoSpec *) n;
+ }
+ | values_in_clause ',' '(' expr_list ')'
+ {
+ PartitionBoundAutoSpec *n = (PartitionBoundAutoSpec *) $1;
+ n->strategy = PARTITION_STRATEGY_LIST;
+ n->listdatumsList = lappend(n->listdatumsList, $4);
+ $$ = (PartitionBoundAutoSpec *) n;
+ }
+ ;
+
+
table_access_method_clause:
USING name { $$ = $2; }
| /*EMPTY*/ { $$ = NULL; }
@@ -15166,6 +15267,7 @@ unreserved_keyword:
| MINUTE_P
| MINVALUE
| MODE
+ | MODULUS
| MONTH_P
| MOVE
| NAME_P
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0e4caa6ad4..e4d638aa04 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -75,6 +75,7 @@
/* State shared by transformCreateStmt and its subroutines */
typedef struct
{
+ CreateStmt *stmt; /* Initial statement */
ParseState *pstate; /* overall parser state */
const char *stmtType; /* "CREATE [FOREIGN] TABLE" or "ALTER TABLE" */
RangeVar *relation; /* relation to create */
@@ -95,6 +96,7 @@ typedef struct
IndexStmt *pkey; /* PRIMARY KEY index, if any */
bool ispartitioned; /* true if table is partitioned */
PartitionBoundSpec *partbound; /* transformed FOR VALUES */
+ PartitionAutoCreate *partautocreate; /* transformed PartitionAutoCreate CONFIGURATION */
bool ofType; /* true if statement contains OF typename */
} CreateStmtContext;
@@ -146,6 +148,7 @@ static Const *transformPartitionBoundValue(ParseState *pstate, Node *con,
const char *colName, Oid colType, int32 colTypmod,
Oid partCollation);
+static void transformPartitionAutoCreate(CreateStmtContext *cxt);
/*
* transformCreateStmt -
@@ -233,6 +236,7 @@ transformCreateStmt(CreateStmt *stmt, const char *queryString)
cxt.stmtType = "CREATE TABLE";
cxt.isforeign = false;
}
+ cxt.stmt = stmt;
cxt.relation = stmt->relation;
cxt.rel = NULL;
cxt.inhRelations = stmt->inhRelations;
@@ -248,6 +252,7 @@ transformCreateStmt(CreateStmt *stmt, const char *queryString)
cxt.pkey = NULL;
cxt.ispartitioned = stmt->partspec != NULL;
cxt.partbound = stmt->partbound;
+ cxt.partautocreate = stmt->partautocreate;
cxt.ofType = (stmt->ofTypename != NULL);
Assert(!stmt->ofTypename || !stmt->inhRelations); /* grammar enforces */
@@ -323,6 +328,10 @@ transformCreateStmt(CreateStmt *stmt, const char *queryString)
*/
transformExtendedStatistics(&cxt);
+ /* Process partition definitions */
+ if (cxt.partautocreate)
+ transformPartitionAutoCreate(&cxt);
+
/*
* Output results.
*/
@@ -4172,3 +4181,160 @@ transformPartitionBoundValue(ParseState *pstate, Node *val,
return (Const *) value;
}
+
+
+/*
+ * Transform configuration into a set of partition bounds.
+ * Generate statements to create partition tables.
+ */
+static void
+transformPartitionAutoCreate(CreateStmtContext *cxt)
+{
+ CreateStmt *part;
+ List *partlist = NIL;
+ ListCell *lc;
+ int i = 0;
+ PartitionBoundAutoSpec *bound = cxt->partautocreate->bound;
+
+ if (cxt->partautocreate->is_deferred)
+ elog(ERROR, "Dynamic generation of partitions is not implemented yet");
+
+ /* Generate regular partbounds based on partautocreate.
+ * Generate create table statements from these partbounds/
+ */
+ if (bound->strategy == PARTITION_STRATEGY_HASH)
+ {
+ for (i = 0; i < bound->modulus; i++)
+ {
+ char *part_relname;
+
+ /*
+ * sGenerate partition name in the format:
+ * $relname_$partnum
+ *
+ * TODO: Add checks on relname length.
+ */
+ part_relname = psprintf("%s_%d", cxt->relation->relname, i);
+
+ part = makeNode(CreateStmt);
+
+ part->relation = makeRangeVar(cxt->relation->schemaname,
+ part_relname, cxt->relation->location);
+ part->tableElts = list_copy(cxt->columns);
+ /* set table as a parent */
+ part->inhRelations = lappend(part->inhRelations, cxt->relation);
+
+ /* Actual partbound generation is here */
+ part->partbound = makeNode(PartitionBoundSpec);
+ part->partbound->strategy = PARTITION_STRATEGY_HASH;
+ part->partbound->modulus = bound->modulus;
+ part->partbound->remainder = i;
+ part->partbound->is_default = false;
+
+ part->partspec = NULL;
+ part->partautocreate = NULL;
+ part->ofTypename = cxt->stmt->ofTypename; //TODO
+ part->constraints = list_copy(cxt->stmt->constraints); //TODO
+ part->options = cxt->stmt->options; //TODO
+ part->oncommit = cxt->stmt->oncommit; //TODO
+ part->tablespacename = cxt->stmt->tablespacename;
+ part->accessMethod = cxt->stmt->accessMethod;
+
+ elog(DEBUG1,"Debug transformPartitionAutoCreate HASH i %d MODULUS %d \n %s\n",
+ i, bound->modulus, nodeToString(part));
+
+ partlist = lappend(partlist, part);
+ }
+ }
+ else if (bound->strategy == PARTITION_STRATEGY_LIST)
+ {
+
+ int n_list_parts = list_length(bound->listdatumsList);
+
+ for (i = 0; i < n_list_parts; i++)
+ {
+ char *part_relname;
+ List *listdatums = (List *)
+ list_nth(bound->listdatumsList, i);
+
+ part_relname = psprintf("%s_%d", cxt->relation->relname, i);
+
+ part = makeNode(CreateStmt);
+
+ part->relation = makeRangeVar(cxt->relation->schemaname,
+ part_relname, cxt->relation->location);
+ part->tableElts = list_copy(cxt->columns);
+ /* set table as a parent */
+ part->inhRelations = lappend(part->inhRelations, cxt->relation);
+
+ /* Actual partbound generation is here */
+ part->partbound = makeNode(PartitionBoundSpec);
+ part->partbound->strategy = PARTITION_STRATEGY_LIST;
+ part->partbound->listdatums = list_copy(listdatums);
+ part->partbound->is_default = false;
+
+ part->partspec = NULL;
+ part->partautocreate = NULL;
+ part->ofTypename = cxt->stmt->ofTypename; //TODO
+ part->constraints = list_copy(cxt->stmt->constraints); //TODO
+ part->options = cxt->stmt->options; //TODO
+ part->oncommit = cxt->stmt->oncommit; //TODO
+ part->tablespacename = cxt->stmt->tablespacename;
+ part->accessMethod = cxt->stmt->accessMethod;
+
+ elog(DEBUG1,"Debug transformPartitionAutoCreate LIST i %d \n %s\n",i, nodeToString(part));
+
+ partlist = lappend(partlist, part);
+ }
+
+ }
+ else if (bound->strategy == PARTITION_STRATEGY_RANGE)
+ {
+
+ elog(WARNING, "Automatic generation of RANGE partition bounds is not implemented yet.\n"
+ "This command will only create one partition");
+ for (i = 0; i < 1; i++)
+ {
+ char *part_relname;
+
+ part_relname = psprintf("%s_%d", cxt->relation->relname, i);
+
+ part = makeNode(CreateStmt);
+
+ part->relation = makeRangeVar(cxt->relation->schemaname,
+ part_relname, cxt->relation->location);
+ part->tableElts = list_copy(cxt->columns);
+ /* set table as a parent */
+ part->inhRelations = lappend(part->inhRelations, cxt->relation);
+
+ /* Actual partbound generation is here */
+ part->partbound = makeNode(PartitionBoundSpec);
+ part->partbound->strategy = PARTITION_STRATEGY_RANGE;
+ /*
+ * TODO Implement partition bound generation:s
+ * add bound->interval to lowerbound, while upperbound is not reached.
+ * Can we use SPI here to simplify operations with different data types
+ * adn their operators?
+ */
+ part->partbound->lowerdatums = list_copy(bound->lowerdatums);
+ part->partbound->upperdatums = list_copy(bound->upperdatums);
+ part->partbound->is_default = false;
+
+ part->partspec = NULL;
+ part->partautocreate = NULL;
+ part->ofTypename = cxt->stmt->ofTypename; //TODO
+ part->constraints = list_copy(cxt->stmt->constraints); //TODO
+ part->options = cxt->stmt->options; //TODO
+ part->oncommit = cxt->stmt->oncommit; //TODO
+ part->tablespacename = cxt->stmt->tablespacename;
+ part->accessMethod = cxt->stmt->accessMethod;
+
+ elog(DEBUG1,"transformPartitionAutoCreate RANGE i %d \n %s\n",i, nodeToString(part));
+
+ partlist = lappend(partlist, part);
+ }
+ }
+
+ /* Add statemets to create each partition */
+ cxt->alist = list_concat(cxt->alist, partlist);
+}
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4..cad0e5e10c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -478,6 +478,8 @@ typedef enum NodeTag
T_PartitionElem,
T_PartitionSpec,
T_PartitionBoundSpec,
+ T_PartitionBoundAutoSpec,
+ T_PartitionAutoCreate,
T_PartitionRangeDatum,
T_PartitionCmd,
T_VacuumRelation,
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 5e1ffafb91..95fc993cb4 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -828,6 +828,48 @@ struct PartitionBoundSpec
int location; /* token location, or -1 if unknown */
};
+/*
+ * PartitionBoundAutoSpec - a partition bound specification
+ *
+ * This represents the rule of generating partition bounds
+ */
+struct PartitionBoundAutoSpec
+{
+ NodeTag type;
+
+ char strategy; /* see PARTITION_STRATEGY codes above */
+
+ /* Partitioning info for HASH strategy: */
+ int modulus;
+
+ /* Partitioning info for LIST strategy: */
+ List *listdatumsList; /* List of lists of Consts (or A_Consts in raw tree) */
+
+ /* Partitioning info for RANGE strategy: */
+ Node *interval; /* TODO */
+ List *lowerdatums; /* List of PartitionRangeDatums */
+ List *upperdatums; /* List of PartitionRangeDatums */
+
+};
+
+/*
+ * PartitionAutoCreate - a partition bound specification for automatic creation
+ *
+ * This represents the information needed automatically calculate partition bounds.
+ * Now only HASH strategy is implemented
+ */
+struct PartitionAutoCreate
+{
+ NodeTag type;
+
+ bool is_deferred; /* create partitions statically (on create statement)
+ * or create them dynamically (when first insertion happens)
+ * DEFERRED creation is not supported yet.
+ */
+ /* The rule of generating partition bounds */
+ PartitionBoundAutoSpec *bound;
+};
+
/*
* PartitionRangeDatum - one of the values in a range partition bound
*
@@ -2075,6 +2117,7 @@ typedef struct CreateStmt
* inhRelation) */
PartitionBoundSpec *partbound; /* FOR VALUES clause */
PartitionSpec *partspec; /* PARTITION BY clause */
+ PartitionAutoCreate *partautocreate; /* CONFIGURATION clause */
TypeName *ofTypename; /* OF typename */
List *constraints; /* constraints (list of Constraint nodes) */
List *options; /* options from WITH clause */
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 08f22ce211..497c58266c 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -251,6 +251,7 @@ PG_KEYWORD("method", METHOD, UNRESERVED_KEYWORD)
PG_KEYWORD("minute", MINUTE_P, UNRESERVED_KEYWORD)
PG_KEYWORD("minvalue", MINVALUE, UNRESERVED_KEYWORD)
PG_KEYWORD("mode", MODE, UNRESERVED_KEYWORD)
+PG_KEYWORD("modulus", MODULUS, UNRESERVED_KEYWORD)
PG_KEYWORD("month", MONTH_P, UNRESERVED_KEYWORD)
PG_KEYWORD("move", MOVE, UNRESERVED_KEYWORD)
PG_KEYWORD("name", NAME_P, UNRESERVED_KEYWORD)
diff --git a/src/include/partitioning/partdefs.h b/src/include/partitioning/partdefs.h
index 6414e2c116..160b1c66ff 100644
--- a/src/include/partitioning/partdefs.h
+++ b/src/include/partitioning/partdefs.h
@@ -19,6 +19,10 @@ typedef struct PartitionKeyData *PartitionKey;
typedef struct PartitionBoundSpec PartitionBoundSpec;
+typedef struct PartitionBoundAutoSpec PartitionBoundAutoSpec;
+
+typedef struct PartitionAutoCreate PartitionAutoCreate;
+
typedef struct PartitionDescData *PartitionDesc;
typedef struct PartitionDirectoryData *PartitionDirectory;
diff --git a/src/test/regress/expected/auto_partitions.out b/src/test/regress/expected/auto_partitions.out
new file mode 100644
index 0000000000..e69de29bb2
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..6ca08bf544 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -119,5 +119,7 @@ test: event_trigger
# this test also uses event triggers, so likewise run it by itself
test: fast_default
+test: auto_partitions
+
# run stats by itself because its delay may be insufficient under heavy load
test: stats
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 979d926119..94da936c84 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,4 +200,5 @@ test: tuplesort
test: explain
test: event_trigger
test: fast_default
+test: auto_partitions
test: stats
diff --git a/src/test/regress/sql/auto_partitions.sql b/src/test/regress/sql/auto_partitions.sql
new file mode 100644
index 0000000000..6df1bbaf48
--- /dev/null
+++ b/src/test/regress/sql/auto_partitions.sql
@@ -0,0 +1,43 @@
+/* Dynamic generation of partitions is not implemented yet */
+CREATE TABLE tbl_hash (i int) PARTITION BY HASH (i)
+CONFIGURATION DEFERRED USING MODULUS 3;
+
+/* Hash */
+CREATE TABLE tbl_hash (i int) PARTITION BY HASH (i)
+CONFIGURATION IMMEDIATE USING MODULUS 3;
+
+\d+ tbl_hash
+
+INSERT INTO tbl_hash select * from generate_series(0,10);
+
+SELECT i from tbl_hash_0;
+SELECT i from tbl_hash_1;
+SELECT i from tbl_hash_2;
+
+DROP TABLE tbl_hash;
+
+/* List */
+CREATE TABLE tbl_list (i char) PARTITION BY LIST (i)
+CONFIGURATION IMMEDIATE USING VALUES IN ('a', 'b'), ('c', 'd'), ('e','f');
+
+\d+ tbl_list
+
+INSERT INTO tbl_list values ('a'), ('b'), ('c'), ('d'), ('e'), ('f');
+
+SELECT i from tbl_list_0;
+SELECT i from tbl_list_1;
+SELECT i from tbl_list_2;
+
+/* Must fail. No default partition */
+INSERT INTO tbl_list values ('q');
+
+DROP TABLE tbl_list;
+
+/* Range.
+ * Automatic generation of RANGE partition bounds is not implemented yet.
+ * This command will only create one partition
+ */
+CREATE TABLE tbl_range (i timestamptz) PARTITION BY RANGE (i)
+ CONFIGURATION IMMEDIATE USING INTERVAL '1 month 2 days' FROM ('2020-01-01') TO ('2020-12-31');
+
+\d+ tbl_range
\ No newline at end of file
--
2.17.1
