Hi,
On 2019/03/01 22:01, Amit Langote wrote:
> Please find attached updated patches. Will update source code comments,
> commit message and perform other fine-tuning over the weekend if possible.
I realized when "fine-tuning" that the patch 0001 contained too many
changes that seem logically separable. I managed to divide it into the
following patches, which also amounts to a much shorter overall diff
against the master. Also, the smaller individual patches made it easier
to spot a lot of useless diffs of inherit.c.
Attached patches are as follows:
1. Create the "otherrel" RelOptInfos of the appendrels as a separate step
of query_planner. Newly added function add_other_rels_to_query() which
adds "other rel" RelOptInfos of child relations is called after
query_planner has finished distributing restrict clauses to baserels.
Child relations in this case include both those of flattened UNION ALL
subqueries and inheritance child tables. Child RangeTblEntrys and
AppendRelInfos are added early in subquery_planner for both types of child
relations. Of the two, we'd like to delay adding the inheritance
children, which is done in the next patch.
See patch 0001.
2. Defer inheritance expansion to add_other_rels_to_query(). Although the
purpose of doing this is to perform partition pruning before adding the
children, this patch doesn't change when the pruning occurs. It deals
with other issues that must be taken care of due to adding children during
query_planner instead of during subquery_planner. Especially,
inheritance_planner now has to add the child target relations on its own.
Also, delaying adding children also affects adding junk columns to the
query's targetlist based on PlanRowMarks, because preprocess_targetlist
can no longer finalize which junk columns to add for a "parent"
PlanRowMark; that must be delayed until all child PlanRowMarks are added
and their allMarkTypes propagated to the parent PlanRowMark.
See patch 0002.
3. Because inheritance_planner calls query_planner separately for each
target child relation and the patch 0002 above puts query_planner in
charge of inheritance expansion, that means child tables of source
inheritance sets will be added as many times as there are target children.
This makes update/delete queries containing inherited source tables
somewhat slow. This patch adjusts inheritance_planner to reuse source
inheritance children added during the planning of 1st child query for the
planning of subsequent child queries.
See patch 0003.
4. Now that all the issues arising due to late inheritance expansion have
been taken care of, this patch moves where partition pruning occurs.
Today it's in set_append_rel_size() and this patch moves it to
expand_partitioned_rtentry() where partitions are added. Only the
partitions remaining after pruning are added, so some members of part_rels
can remain NULL. Some of the places that access partition RelOptInfos
using that array needed to be made aware of that.
See patch 0004.
5. There are a few places which loop over *all* members of part_rels array
of a partitioned parent's RelOptInfo to do something with the partition
rels. Some of those loops run even for point-lookup queries where only
one partition would be accessed, which is inefficient. This patch adds a
Bitmapset member named 'live_parts' to RelOptInfo, whose value is the set
of indexes of unpruned partitions in the parent's RelOptInfo. The
aforementioned loops are now replaced by looping over live_parts Bitmapset
instead.
See patch 0005.
6. set_relation_partition_info today copies the PartitionBoundInfo from
the relcache using partition_bounds_copy. Doing partition_bounds_copy
gets expensive as the number of partitions increases and it really doesn't
seem necessary for the planner to create its own copy. This patch removes
the partition_bounds_copy() and simply uses the relcache pointer.
See patch 0006.
Hope that the above division makes the changes easier to review.
Thanks,
Amit
From 1d83debe07040a93721f17785d075d26dab6f088 Mon Sep 17 00:00:00 2001
From: Amit <amitlangot...@gmail.com>
Date: Sat, 2 Mar 2019 14:13:13 +0900
Subject: [PATCH v26 1/6] Build "other rels" of appendrel baserels in a
separate step
add_other_rels_to_query() runs after query_planner has distributed
restrict clauses to base relations. This will allow us to use
the clauses applied a given partitioned baserel to perform partition
pruning, and add other rels for only the unpruned partiitons.
---
src/backend/optimizer/path/allpaths.c | 2 +-
src/backend/optimizer/plan/initsplan.c | 53 +++++++++++++--
src/backend/optimizer/plan/planmain.c | 15 +++--
src/backend/optimizer/util/relnode.c | 118 +++++++++++++++++++++------------
src/include/optimizer/pathnode.h | 2 +
src/include/optimizer/planmain.h | 1 +
6 files changed, 140 insertions(+), 51 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c
b/src/backend/optimizer/path/allpaths.c
index 0debac75c6..8d8a8f17d5 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -1028,7 +1028,7 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
/*
* The child rel's RelOptInfo was already created during
- * add_base_rels_to_query.
+ * add_other_rels_to_query.
*/
childrel = find_base_rel(root, childRTindex);
Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL);
diff --git a/src/backend/optimizer/plan/initsplan.c
b/src/backend/optimizer/plan/initsplan.c
index 2afc3f1dfe..54fc4dda97 100644
--- a/src/backend/optimizer/plan/initsplan.c
+++ b/src/backend/optimizer/plan/initsplan.c
@@ -20,6 +20,7 @@
#include "nodes/nodeFuncs.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/inherit.h"
#include "optimizer/joininfo.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
@@ -30,6 +31,7 @@
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "parser/analyze.h"
+#include "parser/parsetree.h"
#include "rewrite/rewriteManip.h"
#include "utils/lsyscache.h"
@@ -97,10 +99,11 @@ static void check_hashjoinable(RestrictInfo *restrictinfo);
* jtnode. Internally, the function recurses through the jointree.
*
* At the end of this process, there should be one baserel RelOptInfo for
- * every non-join RTE that is used in the query. Therefore, this routine
- * is the only place that should call build_simple_rel with reloptkind
- * RELOPT_BASEREL. (Note: build_simple_rel recurses internally to build
- * "other rel" RelOptInfos for the members of any appendrels we find here.)
+ * every non-join RTE that is specified in the query. Therefore, this
+ * routine is the only place that should call build_simple_rel with
+ * reloptkind RELOPT_BASEREL. (Note: "other rel" RelOptInfos for the
+ * members of any appendrels we find here are added built later when
+ * query_planner calls add_other_rels_to_query().)
*/
void
add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
@@ -133,6 +136,48 @@ add_base_rels_to_query(PlannerInfo *root, Node *jtnode)
(int) nodeTag(jtnode));
}
+/*
+ * add_other_rels_to_query
+ *
+ * Scan the query's jointree and for each base rels that is an appendrel,
+ * create otherrel RelOptInfos of its children
+ *
+ * At the end of this process, there should be RelOptInfos for all relations
+ * that will be scanned by the query.
+ */
+void
+add_other_rels_to_query(PlannerInfo *root, Node *jtnode)
+{
+ if (jtnode == NULL)
+ return;
+ if (IsA(jtnode, RangeTblRef))
+ {
+ int varno = ((RangeTblRef *)
jtnode)->rtindex;
+ RangeTblEntry *rte = rt_fetch(varno, root->parse->rtable);
+
+ if (rte->inh)
+ (void) add_appendrel_other_rels(root, rte, varno);
+ }
+ else if (IsA(jtnode, FromExpr))
+ {
+ FromExpr *f = (FromExpr *) jtnode;
+ ListCell *l;
+
+ foreach(l, f->fromlist)
+ add_other_rels_to_query(root, lfirst(l));
+ }
+ else if (IsA(jtnode, JoinExpr))
+ {
+ JoinExpr *j = (JoinExpr *) jtnode;
+
+ add_other_rels_to_query(root, j->larg);
+ add_other_rels_to_query(root, j->rarg);
+ }
+ else
+ elog(ERROR, "unrecognized node type: %d",
+ (int) nodeTag(jtnode));
+}
+
/*****************************************************************************
*
diff --git a/src/backend/optimizer/plan/planmain.c
b/src/backend/optimizer/plan/planmain.c
index 3cedd01c98..03c81772a3 100644
--- a/src/backend/optimizer/plan/planmain.c
+++ b/src/backend/optimizer/plan/planmain.c
@@ -159,10 +159,8 @@ query_planner(PlannerInfo *root, List *tlist,
setup_append_rel_array(root);
/*
- * Construct RelOptInfo nodes for all base relations in query, and
- * indirectly for all appendrel member relations ("other rels"). This
- * will give us a RelOptInfo for every "simple" (non-join) rel involved
in
- * the query.
+ * Construct RelOptInfo nodes for all base relations directly mentioned
+ * in query, but not any appendrel member relations ("other rels") yet.
*
* Note: the reason we find the rels by searching the jointree and
* appendrel list, rather than just scanning the rangetable, is that the
@@ -204,6 +202,15 @@ query_planner(PlannerInfo *root, List *tlist,
generate_base_implied_equalities(root);
/*
+ * Now that we have restrict clauses figured out and assigned to proper
+ * base rels, we can proceed to add otherrels, that is, UNION ALL child
+ * tables, inheritance child tables. Having restrict clauses ready
helps
+ * to exclude any children that wouldn't be necessary to scan, based on
+ * constraint exclusion and partition pruning.
+ */
+ add_other_rels_to_query(root, (Node *) root->parse->jointree);
+
+ /*
* We have completed merging equivalence sets, so it's now possible to
* generate pathkeys in canonical form; so compute query_pathkeys and
* other pathkeys fields in PlannerInfo.
diff --git a/src/backend/optimizer/util/relnode.c
b/src/backend/optimizer/util/relnode.c
index 4130514952..a6c3eedc56 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -16,19 +16,26 @@
#include <limits.h>
+#include "access/table.h"
#include "miscadmin.h"
+#include "nodes/makefuncs.h"
#include "optimizer/appendinfo.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
+#include "optimizer/inherit.h"
+#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
#include "optimizer/placeholder.h"
#include "optimizer/plancat.h"
+#include "optimizer/planmain.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "optimizer/tlist.h"
#include "partitioning/partbounds.h"
+#include "partitioning/partdesc.h"
#include "utils/hsearch.h"
+#include "utils/rel.h"
typedef struct JoinHashEntry
@@ -273,53 +280,80 @@ build_simple_rel(PlannerInfo *root, int relid, RelOptInfo
*parent)
root->qual_security_level = Max(root->qual_security_level,
list_length(rte->securityQuals));
+ return rel;
+}
+
+/*
+ * add_appendrel_other_rels
+ * This adds the "other rel" RelOptInfos a given "appendrel"
baserel
+ *
+ * Parent relation in this case is the parent subquery in the flattened UNION
+ * ALL case or an inheritance parent table.
+ */
+void
+add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry *rte, Index rti)
+{
+ ListCell *l;
+ RelOptInfo *rel;
+ Relation relation = NULL;
+ int i;
+
+ rel = find_base_rel(root, rti);
+
/*
- * If this rel is an appendrel parent, recurse to build "other rel"
- * RelOptInfos for its children. They are "other rels" because they are
- * not in the main join tree, but we will need RelOptInfos to plan
access
- * to them.
+ * For partitioned tables, we need to store the child RelOptInfos in the
+ * rel->part_rels array too.
*/
- if (rte->inh)
+ if (rel->part_scheme)
{
- ListCell *l;
- int nparts = rel->nparts;
- int cnt_parts = 0;
-
- if (nparts > 0)
- rel->part_rels = (RelOptInfo **)
- palloc(sizeof(RelOptInfo *) * nparts);
-
- foreach(l, root->append_rel_list)
- {
- AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(l);
- RelOptInfo *childrel;
-
- /* append_rel_list contains all append rels; ignore
others */
- if (appinfo->parent_relid != relid)
- continue;
-
- childrel = build_simple_rel(root, appinfo->child_relid,
-
rel);
-
- /* Nothing more to do for an unpartitioned table. */
- if (!rel->part_scheme)
- continue;
-
- /*
- * The order of partition OIDs in append_rel_list is
the same as
- * the order in the PartitionDesc, so the order of
part_rels will
- * also match the PartitionDesc. See
expand_partitioned_rtentry.
- */
- Assert(cnt_parts < nparts);
- rel->part_rels[cnt_parts] = childrel;
- cnt_parts++;
- }
-
- /* We should have seen all the child partitions. */
- Assert(cnt_parts == nparts);
+ rel->part_rels = (RelOptInfo **)
+ palloc0(sizeof(RelOptInfo *) * rel->nparts);
+ relation = table_open(rte->relid, NoLock);
}
- return rel;
+ /*
+ * We don't need to use expand_planner_arrays in this case, because
+ * no new child RTEs are created. setup_simple_rel_arrays() and
+ * setup_append_rel_array would've considered these child RTEs when
+ * allocating space for various arrays.
+ */
+ i = 0;
+ foreach(l, root->append_rel_list)
+ {
+ AppendRelInfo *appinfo = lfirst(l);
+ Index childRTindex = appinfo->child_relid;
+ RangeTblEntry *childrte;
+ RelOptInfo *childrel;
+
+ if (appinfo->parent_relid != rti)
+ continue;
+
+ Assert(childRTindex < root->simple_rel_array_size);
+ childrte = root->simple_rte_array[childRTindex];
+ Assert(childrte != NULL);
+ childrel = build_simple_rel(root, childRTindex, rel);
+
+ /*
+ * For partitioned parents, we also need to add childrel to its
+ * part_rels array. The order in which child tables appear in
+ * append_rel_list is same as the order in which they appear in
the
+ * parent's PartitionDesc, so assigning partitions like this
works.
+ */
+ if (rel->part_scheme != NULL)
+ {
+ Assert(rel->nparts > 0 && i < rel->nparts);
+ rel->part_rels[i] = childrel;
+ }
+
+ i++;
+
+ /* Child may itself be an inherited relation. */
+ if (childrte->inh)
+ add_appendrel_other_rels(root, childrte, childRTindex);
+ }
+
+ if (relation)
+ table_close(relation, NoLock);
}
/*
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 574bb85b50..1a07963a7d 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -279,6 +279,8 @@ extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
+extern void add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry *rte,
+ Index rti);
extern RelOptInfo *find_base_rel(PlannerInfo *root, int relid);
extern RelOptInfo *find_join_rel(PlannerInfo *root, Relids relids);
extern RelOptInfo *build_join_rel(PlannerInfo *root,
diff --git a/src/include/optimizer/planmain.h b/src/include/optimizer/planmain.h
index 3bbdb5e2f7..035caac500 100644
--- a/src/include/optimizer/planmain.h
+++ b/src/include/optimizer/planmain.h
@@ -65,6 +65,7 @@ extern int from_collapse_limit;
extern int join_collapse_limit;
extern void add_base_rels_to_query(PlannerInfo *root, Node *jtnode);
+extern void add_other_rels_to_query(PlannerInfo *root, Node *jtnode);
extern void build_base_rel_tlists(PlannerInfo *root, List *final_tlist);
extern void add_vars_to_targetlist(PlannerInfo *root, List *vars,
Relids where_needed, bool
create_new_ph);
--
2.11.0
From f6d33752dd38c02707da9149f874b348df956ff0 Mon Sep 17 00:00:00 2001
From: amit <amitlangot...@gmail.com>
Date: Mon, 4 Mar 2019 15:10:29 +0900
Subject: [PATCH v26 2/6] Delay adding inheritance child tables until
query_planner
Inheritance child tables are now added by add_other_rels_to_query,
which is called by query_planner. This replaces
expand_inherited_tables called by subquery_planner as the method
of adding child tables. At the point when add_other_rels_to_query
is called, restriction clauses have already been assigned to
individual base relations, so it will now be possible to perform
partition pruning so that we know exact child tables to add in that
case. This commit however doesn't change when pruning is performed.
Since we're now adding children (and so any PlanRowMarks for them)
much later, preprocess_targetlist cannot conclude which junk columns
to add for a given "parent" PlanRowMark, because it depends on the
value of allMarkTypes field of PlanRowMarks. The correct value of
it cannot be determined until after we've seen all the child tables
that will be scanned, so we must delay adding junk columns based
on "parent" PlanRowMarks too.
inheritance_planner now adds the child target tables on its own,
because subquery_planner doesn't. Also, since it calls query_planner
once for every child relation, source inheritance child tables are
added multiple times. This both slows down queries when there are
inherited tables in UPDATE/DELETE's FROM clause (source inheritance)
and changes the EXPLAIN output a bit because source inheritance
child tables' aliases come out different for every targer child
subplan. Another patch will fix both these issues.
---
contrib/postgres_fdw/expected/postgres_fdw.out | 24 ++---
src/backend/optimizer/plan/planner.c | 54 +++++++---
src/backend/optimizer/prep/preptlist.c | 131 ++++++++++++++++---------
src/backend/optimizer/util/inherit.c | 126 ++++++++++++++++--------
src/backend/optimizer/util/plancat.c | 3 +-
src/backend/optimizer/util/relnode.c | 55 +++++++++++
src/include/nodes/pathnodes.h | 7 ++
src/include/optimizer/inherit.h | 4 +-
src/include/optimizer/pathnode.h | 1 +
src/include/optimizer/prep.h | 2 +
src/test/regress/expected/partition_prune.out | 12 +--
src/test/regress/expected/rowsecurity.out | 16 +--
12 files changed, 298 insertions(+), 137 deletions(-)
diff --git a/contrib/postgres_fdw/expected/postgres_fdw.out
b/contrib/postgres_fdw/expected/postgres_fdw.out
index 42108bd3d4..4d31cfed5d 100644
--- a/contrib/postgres_fdw/expected/postgres_fdw.out
+++ b/contrib/postgres_fdw/expected/postgres_fdw.out
@@ -7128,15 +7128,15 @@ select * from bar where f1 in (select f1 from foo) for
update;
Output: bar2.f1, bar2.f2, bar2.ctid, bar2.*, bar2.tableoid
Remote SQL: SELECT f1, f2, f3, ctid FROM public.loct2 FOR
UPDATE
-> Hash
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
-> HashAggregate
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
Group Key: foo.f1
-> Append
-> Seq Scan on public.foo
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
-> Foreign Scan on public.foo2
- Output: foo2.ctid, foo2.*, foo2.tableoid,
foo2.f1
+ Output: foo2.f1, foo2.ctid, foo2.*,
foo2.tableoid
Remote SQL: SELECT f1, f2, f3, ctid FROM
public.loct1
(23 rows)
@@ -7166,15 +7166,15 @@ select * from bar where f1 in (select f1 from foo) for
share;
Output: bar2.f1, bar2.f2, bar2.ctid, bar2.*, bar2.tableoid
Remote SQL: SELECT f1, f2, f3, ctid FROM public.loct2 FOR
SHARE
-> Hash
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
-> HashAggregate
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
Group Key: foo.f1
-> Append
-> Seq Scan on public.foo
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
-> Foreign Scan on public.foo2
- Output: foo2.ctid, foo2.*, foo2.tableoid,
foo2.f1
+ Output: foo2.f1, foo2.ctid, foo2.*,
foo2.tableoid
Remote SQL: SELECT f1, f2, f3, ctid FROM
public.loct1
(23 rows)
@@ -7203,15 +7203,15 @@ update bar set f2 = f2 + 100 where f1 in (select f1
from foo);
-> Seq Scan on public.bar
Output: bar.f1, bar.f2, bar.ctid
-> Hash
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
-> HashAggregate
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
Group Key: foo.f1
-> Append
-> Seq Scan on public.foo
- Output: foo.ctid, foo.*, foo.tableoid, foo.f1
+ Output: foo.f1, foo.ctid, foo.*, foo.tableoid
-> Foreign Scan on public.foo2
- Output: foo2.ctid, foo2.*, foo2.tableoid,
foo2.f1
+ Output: foo2.f1, foo2.ctid, foo2.*,
foo2.tableoid
Remote SQL: SELECT f1, f2, f3, ctid FROM
public.loct1
-> Hash Join
Output: bar2.f1, (bar2.f2 + 100), bar2.f3, bar2.ctid, foo.ctid,
foo.*, foo.tableoid
diff --git a/src/backend/optimizer/plan/planner.c
b/src/backend/optimizer/plan/planner.c
index bc81535905..c71a40e6ba 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -25,6 +25,7 @@
#include "access/table.h"
#include "access/xact.h"
#include "catalog/pg_constraint.h"
+#include "catalog/pg_inherits.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
#include "executor/executor.h"
@@ -641,6 +642,7 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
root->wt_param_id = -1;
root->non_recursive_path = NULL;
root->partColsUpdated = false;
+ root->contains_inherit_children = false;
/*
* If there is a WITH list, process each WITH query and either convert
it
@@ -714,27 +716,24 @@ subquery_planner(PlannerGlobal *glob, Query *parse,
}
if (rte->lateral)
root->hasLateralRTEs = true;
+
+ /*
+ * While at it, also update the inh status. If the relation
doesn't
+ * or can't have any children, there is no point in letting inh
be set
+ * to true. Note that we do this before processing rowmarks,
so that
+ * the correct information for setting isParent field of
PlanRowMarks.
+ */
+ if (rte->rtekind == RTE_RELATION)
+ rte->inh = rte->inh && has_subclass(rte->relid);
}
/*
* Preprocess RowMark information. We need to do this after subquery
- * pullup (so that all non-inherited RTEs are present) and before
- * inheritance expansion (so that the info is available for
- * expand_inherited_tables to examine and modify).
+ * pullup (so that all non-inherited RTEs are present).
*/
preprocess_rowmarks(root);
/*
- * Expand any rangetable entries that are inheritance sets into "append
- * relations". This can add entries to the rangetable, but they must be
- * plain RTE_RELATION entries, so it's OK (and marginally more
efficient)
- * to do it after checking for joins and other special RTEs. We must do
- * this after pulling up subqueries, else we'd fail to handle inherited
- * tables in subqueries.
- */
- expand_inherited_tables(root);
-
- /*
* Set hasHavingQual to remember if HAVING clause is present. Needed
* because preprocess_expression will reduce a constant-true condition
to
* an empty qual list ... but "HAVING TRUE" is not a semantic no-op.
@@ -1210,10 +1209,31 @@ inheritance_planner(PlannerInfo *root)
Query *parent_parse;
Bitmapset *parent_relids = bms_make_singleton(top_parentRTindex);
PlannerInfo **parent_roots = NULL;
+ List *orig_append_rel_list = list_copy(root->append_rel_list);
Assert(parse->commandType != CMD_INSERT);
/*
+ * Add child target relations. Note that this only adds the children of
+ * the query's target relation and no other. Especially, children of
any
+ * source relations are added when the loop below calls grouping_planner
+ * on the *1st* child target relation.
+ */
+ parent_rte = rt_fetch(top_parentRTindex, root->parse->rtable);
+
+ expand_inherited_rtentry(root, parent_rte, top_parentRTindex);
+
+ /*
+ * If parent no longer has any children, then treat this as an update of
+ * a single table.
+ */
+ if (!parent_rte->inh)
+ {
+ grouping_planner(root, false, 0.0 /* retrieve all tuples */);
+ return;
+ }
+
+ /*
* We generate a modified instance of the original Query for each target
* relation, plan that, and put all the plans into a list that will be
* controlled by a single ModifyTable node. All the instances share the
@@ -1274,7 +1294,6 @@ inheritance_planner(PlannerInfo *root)
* not appear anywhere else in the plan, so the confusion explained
below
* for non-partitioning inheritance cases is not possible.
*/
- parent_rte = rt_fetch(top_parentRTindex, root->parse->rtable);
if (parent_rte->relkind == RELKIND_PARTITIONED_TABLE)
{
nominalRelation = top_parentRTindex;
@@ -1325,6 +1344,9 @@ inheritance_planner(PlannerInfo *root)
subroot = makeNode(PlannerInfo);
memcpy(subroot, parent_root, sizeof(PlannerInfo));
+ /* grouping_planner doesn't need to see the target children. */
+ subroot->append_rel_list = list_copy(orig_append_rel_list);
+
/*
* Generate modified query with this rel as target. We first
apply
* adjust_appendrel_attrs, which copies the Query and changes
@@ -2618,7 +2640,7 @@ preprocess_rowmarks(PlannerInfo *root)
newrc->allMarkTypes = (1 << newrc->markType);
newrc->strength = rc->strength;
newrc->waitPolicy = rc->waitPolicy;
- newrc->isParent = false;
+ newrc->isParent = rte->inh;
prowmarks = lappend(prowmarks, newrc);
}
@@ -2643,7 +2665,7 @@ preprocess_rowmarks(PlannerInfo *root)
newrc->allMarkTypes = (1 << newrc->markType);
newrc->strength = LCS_NONE;
newrc->waitPolicy = LockWaitBlock; /* doesn't matter */
- newrc->isParent = false;
+ newrc->isParent = rte->rtekind == RTE_RELATION ? rte->inh :
false;
prowmarks = lappend(prowmarks, newrc);
}
diff --git a/src/backend/optimizer/prep/preptlist.c
b/src/backend/optimizer/prep/preptlist.c
index 5392d1a561..ea63b6a496 100644
--- a/src/backend/optimizer/prep/preptlist.c
+++ b/src/backend/optimizer/prep/preptlist.c
@@ -126,61 +126,23 @@ preprocess_targetlist(PlannerInfo *root)
foreach(lc, root->rowMarks)
{
PlanRowMark *rc = (PlanRowMark *) lfirst(lc);
- Var *var;
- char resname[32];
- TargetEntry *tle;
+ List *junk_tles;
/* child rels use the same junk attrs as their parents */
if (rc->rti != rc->prti)
continue;
- if (rc->allMarkTypes & ~(1 << ROW_MARK_COPY))
- {
- /* Need to fetch TID */
- var = makeVar(rc->rti,
-
SelfItemPointerAttributeNumber,
- TIDOID,
- -1,
- InvalidOid,
- 0);
- snprintf(resname, sizeof(resname), "ctid%u",
rc->rowmarkId);
- tle = makeTargetEntry((Expr *) var,
-
list_length(tlist) + 1,
-
pstrdup(resname),
- true);
- tlist = lappend(tlist, tle);
- }
- if (rc->allMarkTypes & (1 << ROW_MARK_COPY))
- {
- /* Need the whole row as a junk var */
- var = makeWholeRowVar(rt_fetch(rc->rti, range_table),
- rc->rti,
- 0,
- false);
- snprintf(resname, sizeof(resname), "wholerow%u",
rc->rowmarkId);
- tle = makeTargetEntry((Expr *) var,
-
list_length(tlist) + 1,
-
pstrdup(resname),
- true);
- tlist = lappend(tlist, tle);
- }
+ /*
+ * For inheritance parent row marks, we defer adding junk
columns
+ * until we've added child row marks, because some children
might
+ * require different row mark types which will change the
parent row
+ * mark's allMarkTypes fields.
+ */
+ if (rc->isParent && !root->contains_inherit_children)
+ continue;
- /* If parent of inheritance tree, always fetch the tableoid
too. */
- if (rc->isParent)
- {
- var = makeVar(rc->rti,
- TableOidAttributeNumber,
- OIDOID,
- -1,
- InvalidOid,
- 0);
- snprintf(resname, sizeof(resname), "tableoid%u",
rc->rowmarkId);
- tle = makeTargetEntry((Expr *) var,
-
list_length(tlist) + 1,
-
pstrdup(resname),
- true);
- tlist = lappend(tlist, tle);
- }
+ junk_tles = get_rowmark_junk_tles(root, tlist, rc);
+ tlist = list_concat(tlist, junk_tles);
}
/*
@@ -434,3 +396,74 @@ get_plan_rowmark(List *rowmarks, Index rtindex)
}
return NULL;
}
+
+/*
+ * get_rowmark_junk_tles
+ * Returns TLEs for junk columns necessary for implementing given
+ * PlanRowMark
+ *
+ * These TLEs are needed for locking of rels selected FOR UPDATE/SHARE, and
+ * to do EvalPlanQual rechecking. See comments for PlanRowMark.
+ */
+List *
+get_rowmark_junk_tles(PlannerInfo *root, List *tlist, PlanRowMark *rc)
+{
+ List *range_table = root->parse->rtable;
+ int tlist_len = list_length(tlist);
+ List *junk_tles = NIL;
+ Var *var;
+ char resname[32];
+ TargetEntry *tle;
+
+ if (rc->allMarkTypes & ~(1 << ROW_MARK_COPY))
+ {
+ /* Need to fetch TID */
+ var = makeVar(rc->rti,
+ SelfItemPointerAttributeNumber,
+ TIDOID,
+ -1,
+ InvalidOid,
+ 0);
+ snprintf(resname, sizeof(resname), "ctid%u", rc->rowmarkId);
+ tle = makeTargetEntry((Expr *) var,
+ tlist_len + 1,
+ pstrdup(resname),
+ true);
+ junk_tles = lappend(junk_tles, tle);
+ tlist_len++;
+ }
+ if (rc->allMarkTypes & (1 << ROW_MARK_COPY))
+ {
+ /* Need the whole row as a junk var */
+ var = makeWholeRowVar(rt_fetch(rc->rti, range_table),
+ rc->rti,
+ 0,
+ false);
+ snprintf(resname, sizeof(resname), "wholerow%u", rc->rowmarkId);
+ tle = makeTargetEntry((Expr *) var,
+ tlist_len + 1,
+ pstrdup(resname),
+ true);
+ junk_tles = lappend(junk_tles, tle);
+ tlist_len++;
+ }
+
+ /* For inheritance cases, always fetch the tableoid too. */
+ if (rc->isParent)
+ {
+ var = makeVar(rc->rti,
+ TableOidAttributeNumber,
+ OIDOID,
+ -1,
+ InvalidOid,
+ 0);
+ snprintf(resname, sizeof(resname), "tableoid%u", rc->rowmarkId);
+ tle = makeTargetEntry((Expr *) var,
+ tlist_len + 1,
+ pstrdup(resname),
+ true);
+ junk_tles = lappend(junk_tles, tle);
+ }
+
+ return junk_tles;
+}
diff --git a/src/backend/optimizer/util/inherit.c
b/src/backend/optimizer/util/inherit.c
index a014a12060..fb2977dfbe 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -21,14 +21,14 @@
#include "miscadmin.h"
#include "optimizer/appendinfo.h"
#include "optimizer/inherit.h"
+#include "optimizer/pathnode.h"
+#include "optimizer/planmain.h"
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "partitioning/partdesc.h"
#include "utils/rel.h"
-static void expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry *rte,
- Index rti);
static void expand_partitioned_rtentry(PlannerInfo *root,
RangeTblEntry *parentrte,
Index parentRTindex,
Relation parentrel,
@@ -45,36 +45,6 @@ static Bitmapset *translate_col_privs(const Bitmapset
*parent_privs,
/*
- * expand_inherited_tables
- * Expand each rangetable entry that represents an inheritance set
- * into an "append relation". At the conclusion of this process,
- * the "inh" flag is set in all and only those RTEs that are append
- * relation parents.
- */
-void
-expand_inherited_tables(PlannerInfo *root)
-{
- Index nrtes;
- Index rti;
- ListCell *rl;
-
- /*
- * expand_inherited_rtentry may add RTEs to parse->rtable. The function
is
- * expected to recursively handle any RTEs that it creates with
inh=true.
- * So just scan as far as the original end of the rtable list.
- */
- nrtes = list_length(root->parse->rtable);
- rl = list_head(root->parse->rtable);
- for (rti = 1; rti <= nrtes; rti++)
- {
- RangeTblEntry *rte = (RangeTblEntry *) lfirst(rl);
-
- expand_inherited_rtentry(root, rte, rti);
- rl = lnext(rl);
- }
-}
-
-/*
* expand_inherited_rtentry
* Check whether a rangetable entry represents an inheritance set.
* If so, add entries for all the child tables to the query's
@@ -94,7 +64,7 @@ expand_inherited_tables(PlannerInfo *root)
* Since a partitioned table is not scanned, it might have only one associated
* AppendRelInfo.
*/
-static void
+void
expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry *rte, Index rti)
{
Oid parentOID;
@@ -103,6 +73,7 @@ expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
LOCKMODE lockmode;
List *inhOIDs;
ListCell *l;
+ RelOptInfo *rel = NULL;
/* Does RT entry allow inheritance? */
if (!rte->inh)
@@ -178,6 +149,17 @@ expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
}
/*
+ * Expand planner arrays for adding the child relations. Can't
do
+ * this if we're not being called from query_planner.
+ */
+ if (root->simple_rel_array_size > 0)
+ {
+ /* Needed when creating child RelOptInfos. */
+ rel = find_base_rel(root, rti);
+ expand_planner_arrays(root, list_length(inhOIDs));
+ }
+
+ /*
* This table has no partitions. Expand any plain inheritance
* children in the order the OIDs were returned by
* find_all_inheritors.
@@ -210,6 +192,10 @@ expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
&appinfos, &childrte,
&childRTindex);
+ /* Create the otherrel RelOptInfo too. */
+ if (rel)
+ (void) build_simple_rel(root, childRTindex,
rel);
+
/* Close child relations, but keep locks */
if (childOID != parentOID)
table_close(newrelation, NoLock);
@@ -230,6 +216,19 @@ expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
}
+ /*
+ * Add junk columns needed by the row mark if any and also add the
+ * relevant expressions to the root parent's reltarget.
+ */
+ if (oldrc)
+ {
+ List *tlist = root->processed_tlist;
+ List *junk_tles = get_rowmark_junk_tles(root, tlist, oldrc);
+
+ build_base_rel_tlists(root, junk_tles);
+ root->processed_tlist = list_concat(root->processed_tlist,
junk_tles);
+ }
+
table_close(oldrelation, NoLock);
}
@@ -247,6 +246,7 @@ expand_partitioned_rtentry(PlannerInfo *root, RangeTblEntry
*parentrte,
RangeTblEntry *childrte;
Index childRTindex;
PartitionDesc partdesc = RelationGetPartitionDesc(parentrel);
+ RelOptInfo *rel = NULL;
check_stack_depth();
@@ -266,6 +266,23 @@ expand_partitioned_rtentry(PlannerInfo *root,
RangeTblEntry *parentrte,
root->partColsUpdated =
has_partition_attrs(parentrel, parentrte->updatedCols,
NULL);
+ /*
+ * Expand planner arrays for adding the child relations. Can't do
+ * this if we're not being called from query_planner.
+ *
+ * We'll need nparts + 1 new slots, because we also store the parent as
+ * a child relation. FIXME: no longer add the parent as child too
+ */
+ if (root->simple_rel_array_size > 0)
+ {
+ rel = find_base_rel(root, parentRTindex);
+ expand_planner_arrays(root, partdesc->nparts + 1);
+ Assert(rel->part_rels == NULL);
+ if (partdesc->nparts > 0)
+ rel->part_rels = (RelOptInfo **) palloc0(rel->nparts *
+
sizeof(RelOptInfo *));
+ }
+
/* First expand the partitioned table itself. */
expand_single_inheritance_child(root, parentrte, parentRTindex,
parentrel,
top_parentrc, parentrel,
@@ -301,8 +318,12 @@ expand_partitioned_rtentry(PlannerInfo *root,
RangeTblEntry *parentrte,
parentrel, top_parentrc, childrel,
appinfos, &childrte, &childRTindex);
+ /* Create the otherrel RelOptInfo too. */
+ if (rel)
+ rel->part_rels[i] = build_simple_rel(root,
childRTindex, rel);
+
/* If this child is itself partitioned, recurse */
- if (childrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ if (childrte->inh)
expand_partitioned_rtentry(root, childrte, childRTindex,
childrel, top_parentrc, lockmode,
appinfos);
@@ -345,7 +366,8 @@ expand_single_inheritance_child(PlannerInfo *root,
RangeTblEntry *parentrte,
Oid childOID = RelationGetRelid(childrel);
RangeTblEntry *childrte;
Index childRTindex;
- AppendRelInfo *appinfo;
+ AppendRelInfo *appinfo = NULL;
+ PartitionDesc childrel_partdesc = RelationGetPartitionDesc(childrel);
/*
* Build an RTE for the child, and attach to query's rangetable list. We
@@ -363,9 +385,14 @@ expand_single_inheritance_child(PlannerInfo *root,
RangeTblEntry *parentrte,
*childrte_p = childrte;
childrte->relid = childOID;
childrte->relkind = childrel->rd_rel->relkind;
- /* A partitioned child will need to be expanded further. */
+ /*
+ * A partitioned child will need to be expanded further, but only if it
+ * actually has partitions. A partitioned table with zero children is
+ * specially handled in set_rel_size().
+ */
if (childOID != parentOID &&
- childrte->relkind == RELKIND_PARTITIONED_TABLE)
+ childrte->relkind == RELKIND_PARTITIONED_TABLE &&
+ childrel_partdesc->nparts > 0)
childrte->inh = true;
else
childrte->inh = false;
@@ -376,13 +403,11 @@ expand_single_inheritance_child(PlannerInfo *root,
RangeTblEntry *parentrte,
*childRTindex_p = childRTindex;
/*
- * We need an AppendRelInfo if paths will be built for the child RTE. If
- * childrte->inh is true, then we'll always need to generate append
paths
- * for it. If childrte->inh is false, we must scan it if it's not a
- * partitioned table; but if it is a partitioned table, then it never
has
- * any data of its own and need not be scanned.
+ * Don't need an AppendRelInfo for duplicate RTEs we create for
+ * partitioned table parent rels.
*/
- if (childrte->relkind != RELKIND_PARTITIONED_TABLE || childrte->inh)
+ if (childrte->relid != parentOID ||
+ childrte->relkind != RELKIND_PARTITIONED_TABLE)
{
appinfo = make_append_rel_info(parentrel, childrel,
parentRTindex, childRTindex);
@@ -409,6 +434,21 @@ expand_single_inheritance_child(PlannerInfo *root,
RangeTblEntry *parentrte,
}
/*
+ * Store the RTE and appinfo in the PlannerInfo, which the caller must
+ * already have allocated space for.
+ */
+ if (root->simple_rel_array_size > 0)
+ {
+ Assert(childRTindex < root->simple_rel_array_size);
+ if (root->simple_rte_array[childRTindex])
+ elog(ERROR, "rel %d already exists", childRTindex);
+ root->simple_rte_array[childRTindex] = childrte;
+ if (root->append_rel_array[childRTindex])
+ elog(ERROR, "child relation %d already exists",
childRTindex);
+ root->append_rel_array[childRTindex] = appinfo;
+ }
+
+ /*
* Build a PlanRowMark if parent is marked FOR UPDATE/SHARE.
*/
if (top_parentrc)
diff --git a/src/backend/optimizer/util/plancat.c
b/src/backend/optimizer/util/plancat.c
index 78a96b4ee2..549af9dc6e 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -2090,7 +2090,8 @@ set_relation_partition_info(PlannerInfo *root, RelOptInfo
*rel,
partkey = RelationGetPartitionKey(relation);
rel->part_scheme = find_partition_scheme(root, relation);
Assert(partdesc != NULL && rel->part_scheme != NULL);
- rel->boundinfo = partition_bounds_copy(partdesc->boundinfo, partkey);
+ if (partdesc->nparts > 0)
+ rel->boundinfo = partition_bounds_copy(partdesc->boundinfo,
partkey);
rel->nparts = partdesc->nparts;
set_baserel_partition_key_exprs(relation, rel);
rel->partition_qual = RelationGetPartitionQual(relation);
diff --git a/src/backend/optimizer/util/relnode.c
b/src/backend/optimizer/util/relnode.c
index a6c3eedc56..1dbf19e71b 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -139,6 +139,50 @@ setup_append_rel_array(PlannerInfo *root)
}
/*
+ * expand_planner_arrays
+ * Expand the PlannerInfo arrays by add_size members and
initialize the
+ * the newly added bytes with zero
+ */
+void
+expand_planner_arrays(PlannerInfo *root, int add_size)
+{
+ int new_size;
+
+ Assert(add_size > 0);
+
+ new_size = root->simple_rel_array_size + add_size;
+
+ /* Expand various arrays and 0-initialize added bytes. */
+ root->simple_rte_array = (RangeTblEntry **)
+
repalloc(root->simple_rte_array,
+
sizeof(RangeTblEntry *) * new_size);
+ MemSet(root->simple_rte_array + root->simple_rel_array_size,
+ 0, sizeof(RangeTblEntry *) * add_size);
+ root->simple_rel_array = (RelOptInfo **)
+
repalloc(root->simple_rel_array,
+
sizeof(RelOptInfo *) * new_size);
+ MemSet(root->simple_rel_array + root->simple_rel_array_size,
+ 0, sizeof(RelOptInfo *) * add_size);
+
+ if (root->append_rel_array)
+ {
+ root->append_rel_array = (AppendRelInfo **)
+
repalloc(root->append_rel_array,
+
sizeof(AppendRelInfo *) * new_size);
+ MemSet(root->append_rel_array + root->simple_rel_array_size,
+ 0, sizeof(AppendRelInfo *) * add_size);
+ }
+ else
+ {
+ root->append_rel_array = (AppendRelInfo **)
+
palloc0(sizeof(AppendRelInfo *) *
+
new_size);
+ }
+
+ root->simple_rel_array_size = new_size;
+}
+
+/*
* build_simple_rel
* Construct a new RelOptInfo for a base relation or 'other' relation.
*/
@@ -298,6 +342,17 @@ add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
Relation relation = NULL;
int i;
+ /*
+ * Add inheritance children to the query if not already done. For child
+ * tables that are themselves partitioned, their children will be added
+ * recursively.
+ */
+ if (rte->rtekind == RTE_RELATION &&
!root->contains_inherit_children)
+ {
+ expand_inherited_rtentry(root, rte, rti);
+ return;
+ }
+
rel = find_base_rel(root, rti);
/*
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index a008ae07da..c2e292e54c 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -348,6 +348,13 @@ struct PlannerInfo
/* Does this query modify any partition key columns? */
bool partColsUpdated;
+
+ /*
+ * Does this PlannerInfo and its Query object contain *all* inheritance
+ * children? If true, the RTEs, the AppendRelInfos, and the
PlanRowMarks
+ * of all the children are assumed to be present.
+ */
+ bool contains_inherit_children;
};
diff --git a/src/include/optimizer/inherit.h b/src/include/optimizer/inherit.h
index d2418f15cf..427c1d0abe 100644
--- a/src/include/optimizer/inherit.h
+++ b/src/include/optimizer/inherit.h
@@ -16,7 +16,7 @@
#include "nodes/pathnodes.h"
-
-extern void expand_inherited_tables(PlannerInfo *root);
+extern void expand_inherited_rtentry(PlannerInfo *root, RangeTblEntry *rte,
+ Index rti);
#endif /* INHERIT_H */
diff --git a/src/include/optimizer/pathnode.h b/src/include/optimizer/pathnode.h
index 1a07963a7d..60361507d2 100644
--- a/src/include/optimizer/pathnode.h
+++ b/src/include/optimizer/pathnode.h
@@ -277,6 +277,7 @@ extern Path *reparameterize_path_by_child(PlannerInfo
*root, Path *path,
*/
extern void setup_simple_rel_arrays(PlannerInfo *root);
extern void setup_append_rel_array(PlannerInfo *root);
+extern void expand_planner_arrays(PlannerInfo *root, int add_size);
extern RelOptInfo *build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent);
extern void add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry *rte,
diff --git a/src/include/optimizer/prep.h b/src/include/optimizer/prep.h
index a9b2c9026c..5979c9885e 100644
--- a/src/include/optimizer/prep.h
+++ b/src/include/optimizer/prep.h
@@ -37,6 +37,8 @@ extern Relids get_relids_for_join(Query *query, int
joinrelid);
extern List *preprocess_targetlist(PlannerInfo *root);
extern PlanRowMark *get_plan_rowmark(List *rowmarks, Index rtindex);
+extern List *get_rowmark_junk_tles(PlannerInfo *root, List *tlist,
+ PlanRowMark *rc);
/*
* prototypes for prepunion.c
diff --git a/src/test/regress/expected/partition_prune.out
b/src/test/regress/expected/partition_prune.out
index 30946f77b6..71942394ba 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2559,16 +2559,16 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a
= ab_a1.a;
Index Cond: (a = 1)
-> Nested Loop (actual rows=1 loops=1)
-> Append (actual rows=1 loops=1)
- -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_1 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_2 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b1_a_idx (actual rows=0
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_1 (actual rows=1
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_2 (actual rows=1
loops=1)
Recheck Cond: (a = 1)
Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b2_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_1 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_2 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
@@ -2580,16 +2580,16 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a
= ab_a1.a;
Index Cond: (a = 1)
-> Nested Loop (actual rows=0 loops=1)
-> Append (actual rows=1 loops=1)
- -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_1 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_3 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b1_a_idx (actual rows=0
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_1 (actual rows=1
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_3 (actual rows=1
loops=1)
Recheck Cond: (a = 1)
Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b2_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_1 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_3 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
diff --git a/src/test/regress/expected/rowsecurity.out
b/src/test/regress/expected/rowsecurity.out
index 2e170497c9..a6a499ed4a 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -1815,26 +1815,26 @@ AND f_leak(t1_1.b) AND f_leak(t1_2.b) RETURNING *,
t1_1, t1_2;
-> Seq Scan on t3 t1_2_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Nested Loop
- Join Filter: (t1_1_1.b = t1_2.b)
+ Join Filter: (t1_1_1.b = t1_2_3.b)
-> Seq Scan on t2 t1_1_1
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Append
- -> Seq Scan on t1 t1_2
+ -> Seq Scan on t1 t1_2_3
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t2 t1_2_1
+ -> Seq Scan on t2 t1_2_4
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t3 t1_2_2
+ -> Seq Scan on t3 t1_2_5
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Nested Loop
- Join Filter: (t1_1_2.b = t1_2.b)
+ Join Filter: (t1_1_2.b = t1_2_6.b)
-> Seq Scan on t3 t1_1_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Append
- -> Seq Scan on t1 t1_2
+ -> Seq Scan on t1 t1_2_6
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t2 t1_2_1
+ -> Seq Scan on t2 t1_2_7
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t3 t1_2_2
+ -> Seq Scan on t3 t1_2_8
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
(37 rows)
--
2.11.0
From 7d8c20357ecd031bda163b830d66141f2c3e7346 Mon Sep 17 00:00:00 2001
From: amit <amitlangot...@gmail.com>
Date: Mon, 4 Mar 2019 15:52:11 +0900
Subject: [PATCH v26 3/6] Adjust inheritance_planner to reuse source child
tables
Set of source inheritance child tables won't change across repeated
planning of the query for different target child relations. So,
note down the RangeTblEntrys, AppendRelInfos, and PlanRowMarks of
the source inheritance child tables after the planning for the first
child table is finished. When planning for the subsequent child
target tables, put the saved objects of source inheritance child
tables into the child PlannerInfos and set contains_inherit_children
to true, so that query_planner doesn't add them again.
This restores the regression test outputs to their original state,
because like before, source inheritance child tables are not added
multiple times.
---
src/backend/optimizer/plan/planner.c | 216 +++++++++++++++++---------
src/test/regress/expected/partition_prune.out | 12 +-
src/test/regress/expected/rowsecurity.out | 16 +-
3 files changed, 157 insertions(+), 87 deletions(-)
diff --git a/src/backend/optimizer/plan/planner.c
b/src/backend/optimizer/plan/planner.c
index c71a40e6ba..94557108e8 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -1206,10 +1206,15 @@ inheritance_planner(PlannerInfo *root)
Index rti;
RangeTblEntry *parent_rte;
PlannerInfo *parent_root;
- Query *parent_parse;
Bitmapset *parent_relids = bms_make_singleton(top_parentRTindex);
PlannerInfo **parent_roots = NULL;
List *orig_append_rel_list = list_copy(root->append_rel_list);
+ List *orig_row_marks = list_copy(root->rowMarks);
+ List *orig_rtable = list_copy(root->parse->rtable);
+ List *rtable_with_target;
+ List *source_appinfos = NIL;
+ List *source_child_rowmarks = NIL;
+ List *source_child_rtes = NIL;
Assert(parse->commandType != CMD_INSERT);
@@ -1233,6 +1238,13 @@ inheritance_planner(PlannerInfo *root)
return;
}
+
+ /*
+ * This one also contains the child target relations, but no other
+ * child relations.
+ */
+ rtable_with_target = list_copy(root->parse->rtable);
+
/*
* We generate a modified instance of the original Query for each target
* relation, plan that, and put all the plans into a list that will be
@@ -1249,10 +1261,13 @@ inheritance_planner(PlannerInfo *root)
* management of the rowMarks list.
*
* To begin with, generate a bitmapset of the relids of the subquery
RTEs.
+ * We use orig_rtable, not rtable_with_target (parse->rtable), because
we
+ * wouldn't need to consider any newly added RTEs as they all must be
+ * RTE_RELATION entries.
*/
subqueryRTindexes = NULL;
rti = 1;
- foreach(lc, parse->rtable)
+ foreach(lc, orig_rtable)
{
RangeTblEntry *rte = lfirst_node(RangeTblEntry, lc);
@@ -1265,26 +1280,24 @@ inheritance_planner(PlannerInfo *root)
* Next, we want to identify which AppendRelInfo items contain
references
* to any of the aforesaid subquery RTEs. These items will need to be
* copied and modified to adjust their subquery references; whereas the
- * other ones need not be touched. It's worth being tense over this
- * because we can usually avoid processing most of the AppendRelInfo
- * items, thereby saving O(N^2) space and time when the target is a
large
- * inheritance tree. We can identify AppendRelInfo items by their
- * child_relid, since that should be unique within the list.
+ * other ones need not be touched. We can assume that all (if any)
+ * entries in orig_append_rel_list contain references to subquery RTEs,
+ * because they correspond to flattneed UNION ALL subqueries.
Especially,
+ * we don't need to bother with those added when adding the child target
+ * relations. We can identify AppendRelInfo items by their child_relid,
+ * since that should be unique within the list.
*/
modifiableARIindexes = NULL;
- if (subqueryRTindexes != NULL)
+ foreach(lc, orig_append_rel_list)
{
- foreach(lc, root->append_rel_list)
- {
- AppendRelInfo *appinfo = lfirst_node(AppendRelInfo, lc);
+ AppendRelInfo *appinfo = lfirst_node(AppendRelInfo, lc);
- if (bms_is_member(appinfo->parent_relid,
subqueryRTindexes) ||
- bms_is_member(appinfo->child_relid,
subqueryRTindexes) ||
- bms_overlap(pull_varnos((Node *)
appinfo->translated_vars),
- subqueryRTindexes))
- modifiableARIindexes =
bms_add_member(modifiableARIindexes,
-
appinfo->child_relid);
- }
+ Assert(bms_is_member(appinfo->parent_relid, subqueryRTindexes)
||
+ bms_is_member(appinfo->child_relid,
subqueryRTindexes) ||
+ bms_overlap(pull_varnos((Node *)
appinfo->translated_vars),
+ subqueryRTindexes));
+ modifiableARIindexes = bms_add_member(modifiableARIindexes,
+
appinfo->child_relid);
}
/*
@@ -1335,7 +1348,6 @@ inheritance_planner(PlannerInfo *root)
*/
parent_root = parent_roots[appinfo->parent_relid];
Assert(parent_root != NULL);
- parent_parse = parent_root->parse;
/*
* We need a working copy of the PlannerInfo so that we can
control
@@ -1344,8 +1356,51 @@ inheritance_planner(PlannerInfo *root)
subroot = makeNode(PlannerInfo);
memcpy(subroot, parent_root, sizeof(PlannerInfo));
- /* grouping_planner doesn't need to see the target children. */
- subroot->append_rel_list = list_copy(orig_append_rel_list);
+ /*
+ * Per the above comment, we'll be changing only the
AppendRelInfos
+ * that are contained in orig_append_rel_list, so only copy
those.
+ */
+ subroot->append_rel_list = copyObject(orig_append_rel_list);
+
+ /*
+ * Likewise, for PlanRowMarks. (Fortunately, the executor
doesn't
+ * need to see the modified copies --- we can just pass it the
+ * original rowMarks list.)
+ */
+ subroot->rowMarks = copyObject(orig_row_marks);
+
+ /*
+ * No need to copy of the RTEs themselves, but do copy the List
+ * structure.
+ */
+ subroot->parse->rtable = list_copy(rtable_with_target);
+
+ /*
+ * If this isn't the first child query, then we can use the
child
+ * objects for source child relations created during the
planning of
+ * 1st child query. IOW, this planning run doesn't need to
create the
+ * child objects again, indicated by setting
contains_inherit_children
+ * sub-PlannerInfo.
+ */
+ if (source_appinfos)
+ {
+ subroot->append_rel_list =
list_concat(subroot->append_rel_list,
+
source_appinfos);
+ /*
+ * XXX Assert(source_child_rowmarks != NIL);
+ */
+ subroot->rowMarks = list_concat(subroot->rowMarks,
+
source_child_rowmarks);
+ Assert(source_child_rtes != NIL);
+ subroot->parse->rtable =
list_concat(subroot->parse->rtable,
+
source_child_rtes);
+
+ /*
+ * We have the children, so no need to add them again
during this
+ * planning cycle.
+ */
+ subroot->contains_inherit_children = true;
+ }
/*
* Generate modified query with this rel as target. We first
apply
@@ -1355,7 +1410,7 @@ inheritance_planner(PlannerInfo *root)
*/
subroot->parse = (Query *)
adjust_appendrel_attrs(parent_root,
- (Node *)
parent_parse,
+ (Node *)
subroot->parse,
1, &appinfo);
/*
@@ -1421,41 +1476,6 @@ inheritance_planner(PlannerInfo *root)
nominalRelation = appinfo->child_relid;
/*
- * The rowMarks list might contain references to subquery RTEs,
so
- * make a copy that we can apply ChangeVarNodes to.
(Fortunately, the
- * executor doesn't need to see the modified copies --- we can
just
- * pass it the original rowMarks list.)
- */
- subroot->rowMarks = copyObject(parent_root->rowMarks);
-
- /*
- * The append_rel_list likewise might contain references to
subquery
- * RTEs (if any subqueries were flattenable UNION ALLs). So
prepare
- * to apply ChangeVarNodes to that, too. As explained above,
we only
- * want to copy items that actually contain such references;
the rest
- * can just get linked into the subroot's append_rel_list.
- *
- * If we know there are no such references, we can just use the
outer
- * append_rel_list unmodified.
- */
- if (modifiableARIindexes != NULL)
- {
- ListCell *lc2;
-
- subroot->append_rel_list = NIL;
- foreach(lc2, parent_root->append_rel_list)
- {
- AppendRelInfo *appinfo2 =
lfirst_node(AppendRelInfo, lc2);
-
- if (bms_is_member(appinfo2->child_relid,
modifiableARIindexes))
- appinfo2 = copyObject(appinfo2);
-
- subroot->append_rel_list =
lappend(subroot->append_rel_list,
-
appinfo2);
- }
- }
-
- /*
* Add placeholders to the child Query's rangetable list to
fill the
* RT indexes already reserved for subqueries in previous
children.
* These won't be referenced, so there's no need to make them
very
@@ -1468,23 +1488,23 @@ inheritance_planner(PlannerInfo *root)
/*
* If this isn't the first child Query, generate duplicates of
all
* subquery RTEs, and adjust Var numbering to reference the
- * duplicates. To simplify the loop logic, we scan the original
rtable
- * not the copy just made by adjust_appendrel_attrs; that
should be OK
- * since subquery RTEs couldn't contain any references to the
target
- * rel.
+ * duplicates. Note that we scan the original rtable before any
+ * child target relations were added, which is OK, because no
other
+ * RTEs would contain references to subquery rels being
modified.
*/
if (final_rtable != NIL && subqueryRTindexes != NULL)
{
ListCell *lr;
rti = 1;
- foreach(lr, parent_parse->rtable)
+ foreach(lr, orig_rtable)
{
RangeTblEntry *rte = lfirst_node(RangeTblEntry,
lr);
if (bms_is_member(rti, subqueryRTindexes))
{
Index newrti;
+ ListCell *lc2;
/*
* The RTE can't contain any references
to its own RT
@@ -1495,19 +1515,21 @@ inheritance_planner(PlannerInfo *root)
newrti =
list_length(subroot->parse->rtable) + 1;
ChangeVarNodes((Node *) subroot->parse,
rti, newrti, 0);
ChangeVarNodes((Node *)
subroot->rowMarks, rti, newrti, 0);
- /* Skip processing unchanging parts of
append_rel_list */
- if (modifiableARIindexes != NULL)
+
+ /*
+ * UNION ALL related appinfos are the
beginning of the
+ * list, only change those.
+ */
+ foreach(lc2, subroot->append_rel_list)
{
- ListCell *lc2;
+ AppendRelInfo *appinfo2 =
lfirst_node(AppendRelInfo,
+
lc2);
- foreach(lc2,
subroot->append_rel_list)
- {
- AppendRelInfo *appinfo2
= lfirst_node(AppendRelInfo, lc2);
-
- if
(bms_is_member(appinfo2->child_relid,
-
modifiableARIindexes))
-
ChangeVarNodes((Node *) appinfo2, rti, newrti, 0);
- }
+ if
(bms_is_member(appinfo2->child_relid,
+
modifiableARIindexes))
+ ChangeVarNodes((Node *)
appinfo2, rti, newrti, 0);
+ else
+ break;
}
rte = copyObject(rte);
ChangeVarNodes((Node *)
rte->securityQuals, rti, newrti, 0);
@@ -1536,6 +1558,54 @@ inheritance_planner(PlannerInfo *root)
subpath = sub_final_rel->cheapest_total_path;
/*
+ * If we finished planning our first child query, copy the
source
+ * child objects that were added during its planning.
+ */
+ if (source_appinfos == NIL && subroot->append_rel_list)
+ {
+ int num_skip_appinfos =
list_length(orig_append_rel_list);
+ int num_skip_rowmarks =
list_length(orig_row_marks);
+ int num_skip_rtes =
list_length(rtable_with_target);
+ ListCell *lc2;
+
+ source_appinfos =
list_copy_tail(subroot->append_rel_list,
+
num_skip_appinfos);
+ Assert(source_child_rowmarks == NIL);
+ source_child_rowmarks =
list_copy_tail(subroot->rowMarks,
+
num_skip_rowmarks);
+ Assert(source_child_rtes == NIL);
+ source_child_rtes =
list_copy_tail(subroot->parse->rtable,
+
num_skip_rtes);
+
+ /*
+ * Original parent PlanRowMark is modified when adding
the
+ * child PlanRowMarks. Copy those changes so that the
planning
+ * of subsequent child queries works correctly. That is
+ * necessary, because we won't be adding child objects
again,
+ * so there won't be an opportunity to modify the parent
+ * PlanRowMark as desired.
+ *
+ * All the original parent row marks are the beginning
of
+ * subroot->rowMarks, skip the rest.
+ */
+ foreach(lc2, orig_row_marks)
+ {
+ PlanRowMark *oldrc = lfirst_node(PlanRowMark,
lc2);
+ ListCell *lc3;
+
+ foreach(lc3, subroot->rowMarks)
+ {
+ PlanRowMark *newrc =
lfirst_node(PlanRowMark, lc3);
+
+ if (oldrc->rti == newrc->rti)
+ oldrc->allMarkTypes =
newrc->allMarkTypes;
+ else
+ break;
+ }
+ }
+ }
+
+ /*
* If this child rel was excluded by constraint exclusion,
exclude it
* from the result plan.
*/
diff --git a/src/test/regress/expected/partition_prune.out
b/src/test/regress/expected/partition_prune.out
index 71942394ba..30946f77b6 100644
--- a/src/test/regress/expected/partition_prune.out
+++ b/src/test/regress/expected/partition_prune.out
@@ -2559,16 +2559,16 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a
= ab_a1.a;
Index Cond: (a = 1)
-> Nested Loop (actual rows=1 loops=1)
-> Append (actual rows=1 loops=1)
- -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_2 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_1 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b1_a_idx (actual rows=0
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_2 (actual rows=1
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_1 (actual rows=1
loops=1)
Recheck Cond: (a = 1)
Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b2_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_2 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_1 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
@@ -2580,16 +2580,16 @@ update ab_a1 set b = 3 from ab where ab.a = 1 and ab.a
= ab_a1.a;
Index Cond: (a = 1)
-> Nested Loop (actual rows=0 loops=1)
-> Append (actual rows=1 loops=1)
- -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_3 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b1 ab_a1_b1_1 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b1_a_idx (actual rows=0
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_3 (actual rows=1
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b2 ab_a1_b2_1 (actual rows=1
loops=1)
Recheck Cond: (a = 1)
Heap Blocks: exact=1
-> Bitmap Index Scan on ab_a1_b2_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
- -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_3 (actual rows=0
loops=1)
+ -> Bitmap Heap Scan on ab_a1_b3 ab_a1_b3_1 (actual rows=0
loops=1)
Recheck Cond: (a = 1)
-> Bitmap Index Scan on ab_a1_b3_a_idx (actual rows=1
loops=1)
Index Cond: (a = 1)
diff --git a/src/test/regress/expected/rowsecurity.out
b/src/test/regress/expected/rowsecurity.out
index a6a499ed4a..2e170497c9 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -1815,26 +1815,26 @@ AND f_leak(t1_1.b) AND f_leak(t1_2.b) RETURNING *,
t1_1, t1_2;
-> Seq Scan on t3 t1_2_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Nested Loop
- Join Filter: (t1_1_1.b = t1_2_3.b)
+ Join Filter: (t1_1_1.b = t1_2.b)
-> Seq Scan on t2 t1_1_1
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Append
- -> Seq Scan on t1 t1_2_3
+ -> Seq Scan on t1 t1_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t2 t1_2_4
+ -> Seq Scan on t2 t1_2_1
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t3 t1_2_5
+ -> Seq Scan on t3 t1_2_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Nested Loop
- Join Filter: (t1_1_2.b = t1_2_6.b)
+ Join Filter: (t1_1_2.b = t1_2.b)
-> Seq Scan on t3 t1_1_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
-> Append
- -> Seq Scan on t1 t1_2_6
+ -> Seq Scan on t1 t1_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t2 t1_2_7
+ -> Seq Scan on t2 t1_2_1
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
- -> Seq Scan on t3 t1_2_8
+ -> Seq Scan on t3 t1_2_2
Filter: ((a = 4) AND ((a % 2) = 0) AND f_leak(b))
(37 rows)
--
2.11.0
From d3fcb2331bfa8101195134f93aaf7e31010d3361 Mon Sep 17 00:00:00 2001
From: amit <amitlangot...@gmail.com>
Date: Mon, 4 Mar 2019 11:43:38 +0900
Subject: [PATCH v26 4/6] Perform pruning in expand_partitioned_rtentry
This allows to avoid opening/locking partitions that won't be
scanned at all. Since expand_partitioned_rtentry recursivly processes
sub-partitioned tables, translated quals must be added right when the
child RelOptInfo is built for the sub-partitioned tables. So,
build_simple_rel itself performs apply_child_basequals.
Code for partitioning optimizations that access partition RelOptInfos
from part_rels array of the parent's RelOptInfo must now be made
aware that some entries might be NULL due partition pruning. In the
case of partitionwise join, even a pruned partition may need to be
joined if it falls on the nullable side of an outer join. A dummy
RelOptInfo containing a dummy path is built in that case and put
into its parent's part_rels array.
---
src/backend/optimizer/path/allpaths.c | 176 +---------------------------------
src/backend/optimizer/path/joinrels.c | 92 +++++++++++++++++-
src/backend/optimizer/plan/planner.c | 8 ++
src/backend/optimizer/util/inherit.c | 11 ++-
src/backend/optimizer/util/relnode.c | 174 ++++++++++++++++++++++++++++++++-
src/backend/partitioning/partprune.c | 43 +++++----
6 files changed, 304 insertions(+), 200 deletions(-)
diff --git a/src/backend/optimizer/path/allpaths.c
b/src/backend/optimizer/path/allpaths.c
index 8d8a8f17d5..3191836c19 100644
--- a/src/backend/optimizer/path/allpaths.c
+++ b/src/backend/optimizer/path/allpaths.c
@@ -138,9 +138,6 @@ static void subquery_push_qual(Query *subquery,
static void recurse_push_qual(Node *setOp, Query *topquery,
RangeTblEntry *rte, Index rti, Node *qual);
static void remove_unused_subquery_outputs(Query *subquery, RelOptInfo *rel);
-static bool apply_child_basequals(PlannerInfo *root, RelOptInfo *rel,
- RelOptInfo *childrel,
- RangeTblEntry *childRTE,
AppendRelInfo *appinfo);
/*
@@ -945,8 +942,6 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
double *parent_attrsizes;
int nattrs;
ListCell *l;
- Relids live_children = NULL;
- bool did_pruning = false;
/* Guard against stack overflow due to overly deep inheritance tree. */
check_stack_depth();
@@ -965,21 +960,6 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
rel->partitioned_child_rels = list_make1_int(rti);
/*
- * If the partitioned relation has any baserestrictinfo quals then we
- * attempt to use these quals to prune away partitions that cannot
- * possibly contain any tuples matching these quals. In this case we'll
- * store the relids of all partitions which could possibly contain a
- * matching tuple, and skip anything else in the loop below.
- */
- if (enable_partition_pruning &&
- rte->relkind == RELKIND_PARTITIONED_TABLE &&
- rel->baserestrictinfo != NIL)
- {
- live_children = prune_append_rel_partitions(rel);
- did_pruning = true;
- }
-
- /*
* If this is a partitioned baserel, set the consider_partitionwise_join
* flag; currently, we only consider partitionwise joins with the
baserel
* if its targetlist doesn't contain a whole-row Var.
@@ -1033,30 +1013,11 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
childrel = find_base_rel(root, childRTindex);
Assert(childrel->reloptkind == RELOPT_OTHER_MEMBER_REL);
- if (did_pruning && !bms_is_member(appinfo->child_relid,
live_children))
- {
- /* This partition was pruned; skip it. */
- set_dummy_rel_pathlist(childrel);
+ /* build_simple_rel may have already proven the child to be
dummy. */
+ if (IS_DUMMY_REL(childrel))
continue;
- }
-
- /*
- * We have to copy the parent's targetlist and quals to the
child,
- * with appropriate substitution of variables. If any constant
false
- * or NULL clauses turn up, we can disregard the child right
away.
- * If not, we can apply constraint exclusion with just the
- * baserestrictinfo quals.
- */
- if (!apply_child_basequals(root, rel, childrel, childRTE,
appinfo))
- {
- /*
- * Some restriction clause reduced to constant FALSE or
NULL after
- * substitution, so this child need not be scanned.
- */
- set_dummy_rel_pathlist(childrel);
- continue;
- }
+ /* Apply constraint exclusion. */
if (relation_excluded_by_constraints(root, childrel, childRTE))
{
/*
@@ -1068,7 +1029,8 @@ set_append_rel_size(PlannerInfo *root, RelOptInfo *rel,
}
/*
- * CE failed, so finish copying/modifying targetlist and join
quals.
+ * Constraint exclusion failed, so finish copying/modifying
targetlist
+ * and join quals.
*
* NB: the resulting childrel->reltarget->exprs may contain
arbitrary
* expressions, which otherwise would not occur in a rel's
targetlist.
@@ -3557,134 +3519,6 @@ generate_partitionwise_join_paths(PlannerInfo *root,
RelOptInfo *rel)
list_free(live_children);
}
-/*
- * apply_child_basequals
- * Populate childrel's quals based on rel's quals, translating
them using
- * appinfo.
- *
- * If any of the resulting clauses evaluate to false or NULL, we return false
- * and don't apply any quals. Caller can mark the relation as a dummy rel in
- * this case, since it needn't be scanned.
- *
- * If any resulting clauses evaluate to true, they're unnecessary and we don't
- * apply then.
- */
-static bool
-apply_child_basequals(PlannerInfo *root, RelOptInfo *rel,
- RelOptInfo *childrel, RangeTblEntry
*childRTE,
- AppendRelInfo *appinfo)
-{
- List *childquals;
- Index cq_min_security;
- ListCell *lc;
-
- /*
- * The child rel's targetlist might contain non-Var expressions, which
- * means that substitution into the quals could produce opportunities
for
- * const-simplification, and perhaps even pseudoconstant quals.
Therefore,
- * transform each RestrictInfo separately to see if it reduces to a
- * constant or pseudoconstant. (We must process them separately to keep
- * track of the security level of each qual.)
- */
- childquals = NIL;
- cq_min_security = UINT_MAX;
- foreach(lc, rel->baserestrictinfo)
- {
- RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
- Node *childqual;
- ListCell *lc2;
-
- Assert(IsA(rinfo, RestrictInfo));
- childqual = adjust_appendrel_attrs(root,
-
(Node *) rinfo->clause,
-
1, &appinfo);
- childqual = eval_const_expressions(root, childqual);
- /* check for flat-out constant */
- if (childqual && IsA(childqual, Const))
- {
- if (((Const *) childqual)->constisnull ||
- !DatumGetBool(((Const *)
childqual)->constvalue))
- {
- /* Restriction reduces to constant FALSE or
NULL */
- return false;
- }
- /* Restriction reduces to constant TRUE, so drop it */
- continue;
- }
- /* might have gotten an AND clause, if so flatten it */
- foreach(lc2, make_ands_implicit((Expr *) childqual))
- {
- Node *onecq = (Node *) lfirst(lc2);
- bool pseudoconstant;
-
- /* check for pseudoconstant (no Vars or volatile
functions) */
- pseudoconstant =
- !contain_vars_of_level(onecq, 0) &&
- !contain_volatile_functions(onecq);
- if (pseudoconstant)
- {
- /* tell createplan.c to check for gating quals
*/
- root->hasPseudoConstantQuals = true;
- }
- /* reconstitute RestrictInfo with appropriate
properties */
- childquals = lappend(childquals,
-
make_restrictinfo((Expr *) onecq,
-
rinfo->is_pushed_down,
-
rinfo->outerjoin_delayed,
-
pseudoconstant,
-
rinfo->security_level,
-
NULL, NULL, NULL));
- /* track minimum security level among child quals */
- cq_min_security = Min(cq_min_security,
rinfo->security_level);
- }
- }
-
- /*
- * In addition to the quals inherited from the parent, we might have
- * securityQuals associated with this particular child node. (Currently
- * this can only happen in appendrels originating from UNION ALL;
- * inheritance child tables don't have their own securityQuals, see
- * expand_inherited_rtentry().) Pull any such securityQuals up into the
- * baserestrictinfo for the child. This is similar to
- * process_security_barrier_quals() for the parent rel, except that we
- * can't make any general deductions from such quals, since they don't
- * hold for the whole appendrel.
- */
- if (childRTE->securityQuals)
- {
- Index security_level = 0;
-
- foreach(lc, childRTE->securityQuals)
- {
- List *qualset = (List *) lfirst(lc);
- ListCell *lc2;
-
- foreach(lc2, qualset)
- {
- Expr *qual = (Expr *) lfirst(lc2);
-
- /* not likely that we'd see constants here, so
no check */
- childquals = lappend(childquals,
-
make_restrictinfo(qual,
-
true, false, false,
-
security_level,
-
NULL, NULL, NULL));
- cq_min_security = Min(cq_min_security,
security_level);
- }
- security_level++;
- }
- Assert(security_level <= root->qual_security_level);
- }
-
- /*
- * OK, we've got all the baserestrictinfo quals for this child.
- */
- childrel->baserestrictinfo = childquals;
- childrel->baserestrict_min_security = cq_min_security;
-
- return true;
-}
-
/*****************************************************************************
* DEBUG SUPPORT
*****************************************************************************/
diff --git a/src/backend/optimizer/path/joinrels.c
b/src/backend/optimizer/path/joinrels.c
index dfbbfdac6d..66a5cf3334 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/table.h"
#include "miscadmin.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/appendinfo.h"
@@ -21,6 +22,8 @@
#include "optimizer/joininfo.h"
#include "optimizer/pathnode.h"
#include "optimizer/paths.h"
+#include "optimizer/tlist.h"
+#include "parser/parsetree.h"
#include "partitioning/partbounds.h"
#include "utils/lsyscache.h"
#include "utils/memutils.h"
@@ -52,6 +55,9 @@ static SpecialJoinInfo *build_child_join_sjinfo(PlannerInfo
*root,
Relids left_relids, Relids
right_relids);
static int match_expr_to_partition_keys(Expr *expr, RelOptInfo *rel,
bool strict_op);
+static RelOptInfo *build_dummy_partition_rel(PlannerInfo *root,
+
RelOptInfo *parent, Relation parentrel,
+ int
partidx);
/*
@@ -1318,6 +1324,8 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo
*rel1, RelOptInfo *rel2,
RelOptInfo *joinrel, SpecialJoinInfo
*parent_sjinfo,
List *parent_restrictlist)
{
+ Relation baserel1 = NULL,
+ baserel2 = NULL;
bool rel1_is_simple = IS_SIMPLE_REL(rel1);
bool rel2_is_simple = IS_SIMPLE_REL(rel2);
int nparts;
@@ -1368,6 +1376,18 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo
*rel1, RelOptInfo *rel2,
nparts = joinrel->nparts;
+ if (rel1_is_simple)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rel1->relid, root);
+
+ baserel1 = table_open(rte->relid, NoLock);
+ }
+ if (rel2_is_simple)
+ {
+ RangeTblEntry *rte = planner_rt_fetch(rel2->relid, root);
+
+ baserel2 = table_open(rte->relid, NoLock);
+ }
/*
* Create child-join relations for this partitioned join, if those don't
* exist. Add paths to child-joins for a pair of child relations
@@ -1384,6 +1404,13 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo
*rel1, RelOptInfo *rel2,
AppendRelInfo **appinfos;
int nappinfos;
+ if (rel1_is_simple && child_rel1 == NULL)
+ child_rel1 = build_dummy_partition_rel(root, rel1,
baserel1,
+
cnt_parts);
+ if (rel2_is_simple && child_rel2 == NULL)
+ child_rel2 = build_dummy_partition_rel(root, rel2,
baserel2,
+
cnt_parts);
+
/*
* If a child table has consider_partitionwise_join=false, it
means
* that it's a dummy relation for which we skipped setting up
tlist
@@ -1444,6 +1471,11 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo
*rel1, RelOptInfo *rel2,
child_joinrel, child_sjinfo,
child_restrictlist);
}
+
+ if (baserel1)
+ table_close(baserel1, NoLock);
+ if (baserel2)
+ table_close(baserel2, NoLock);
}
/*
@@ -1462,8 +1494,14 @@ update_child_rel_info(PlannerInfo *root,
(Node *)
rel->reltarget->exprs,
1, &appinfo);
- /* Make child entries in the EquivalenceClass as well */
- if (rel->has_eclass_joins || has_useful_pathkeys(root, rel))
+ /*
+ * Make child entries in the EquivalenceClass as well. If the childrel
+ * appears to be a dummy one (one built by build_dummy_partition_rel()),
+ * no need to make any new entries, because anything that'd need those
+ * can instead use the parent's (rel).
+ */
+ if (childrel->relid != rel->relid &&
+ (rel->has_eclass_joins || has_useful_pathkeys(root, rel)))
add_child_rel_equivalences(root, appinfo, rel, childrel);
childrel->has_eclass_joins = rel->has_eclass_joins;
}
@@ -1674,3 +1712,53 @@ match_expr_to_partition_keys(Expr *expr, RelOptInfo
*rel, bool strict_op)
return -1;
}
+
+/*
+ * build_dummy_partition_rel
+ * Build a RelOptInfo and AppendRelInfo for a pruned partition
+ *
+ * This does not result in opening the relation or a range table entry being
+ * created. Also, the RelOptInfo thus created is not stored anywhere else
+ * beside the parent's part_rels array.
+ *
+ * The only reason this exists is because partition-wise join, in some cases,
+ * needs a RelOptInfo to represent an empty relation that's on the nullable
+ * side of an outer join, so that a Path representing the outer join can be
+ * created.
+ */
+static RelOptInfo *
+build_dummy_partition_rel(PlannerInfo *root, RelOptInfo *parent,
+ Relation parentrel, int
partidx)
+{
+ RelOptInfo *rel;
+
+ Assert(parent->part_rels[partidx] == NULL);
+
+ /* Create minimally valid-looking RelOptInfo with parent's relid. */
+ rel = makeNode(RelOptInfo);
+ rel->reloptkind = RELOPT_OTHER_MEMBER_REL;
+ rel->relid = parent->relid;
+ rel->relids = bms_copy(parent->relids);
+ if (parent->top_parent_relids)
+ rel->top_parent_relids = parent->top_parent_relids;
+ else
+ rel->top_parent_relids = bms_copy(parent->relids);
+ rel->reltarget = copy_pathtarget(parent->reltarget);
+ parent->part_rels[partidx] = rel;
+ mark_dummy_rel(rel);
+
+ /*
+ * Now we'll need a (no-op) AppendRelInfo for parent, because we're
+ * setting the dummy partition's relid to be same as the parent's.
+ */
+ if (root->append_rel_array[parent->relid] == NULL)
+ {
+ AppendRelInfo *appinfo = make_append_rel_info(parentrel,
parentrel,
+
parent->relid,
+
parent->relid);
+
+ root->append_rel_array[parent->relid] = appinfo;
+ }
+
+ return rel;
+}
diff --git a/src/backend/optimizer/plan/planner.c
b/src/backend/optimizer/plan/planner.c
index 94557108e8..7fd1366652 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7138,6 +7138,10 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
int nappinfos;
List *child_scanjoin_targets = NIL;
+ /* Skip processing pruned partitions. */
+ if (child_rel == NULL)
+ continue;
+
/* Translate scan/join targets for this child. */
appinfos = find_appinfos_by_relids(root,
child_rel->relids,
&nappinfos);
@@ -7239,6 +7243,10 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
RelOptInfo *child_grouped_rel;
RelOptInfo *child_partially_grouped_rel;
+ /* Skip processing pruned partitions. */
+ if (child_input_rel == NULL)
+ continue;
+
/* Input child rel must have a path */
Assert(child_input_rel->pathlist != NIL);
diff --git a/src/backend/optimizer/util/inherit.c
b/src/backend/optimizer/util/inherit.c
index fb2977dfbe..e2c1f341ce 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -26,6 +26,7 @@
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "partitioning/partdesc.h"
+#include "partitioning/partprune.h"
#include "utils/rel.h"
@@ -242,6 +243,7 @@ expand_partitioned_rtentry(PlannerInfo *root, RangeTblEntry
*parentrte,
PlanRowMark *top_parentrc,
LOCKMODE lockmode,
List **appinfos)
{
+ Bitmapset *live_parts = 0;
int i;
RangeTblEntry *childrte;
Index childRTindex;
@@ -276,12 +278,16 @@ expand_partitioned_rtentry(PlannerInfo *root,
RangeTblEntry *parentrte,
if (root->simple_rel_array_size > 0)
{
rel = find_base_rel(root, parentRTindex);
- expand_planner_arrays(root, partdesc->nparts + 1);
+ if (partdesc->nparts > 0)
+ live_parts = prune_append_rel_partitions(rel);
+ expand_planner_arrays(root, bms_num_members(live_parts) + 1);
Assert(rel->part_rels == NULL);
if (partdesc->nparts > 0)
rel->part_rels = (RelOptInfo **) palloc0(rel->nparts *
sizeof(RelOptInfo *));
}
+ else
+ live_parts = bms_add_range(NULL, 0, partdesc->nparts - 1);
/* First expand the partitioned table itself. */
expand_single_inheritance_child(root, parentrte, parentRTindex,
parentrel,
@@ -298,7 +304,8 @@ expand_partitioned_rtentry(PlannerInfo *root, RangeTblEntry
*parentrte,
return;
}
- for (i = 0; i < partdesc->nparts; i++)
+ i = -1;
+ while ((i = bms_next_member(live_parts, i)) >= 0)
{
Oid childOID = partdesc->oids[i];
Relation childrel;
diff --git a/src/backend/optimizer/util/relnode.c
b/src/backend/optimizer/util/relnode.c
index 1dbf19e71b..adc06cf69a 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -70,6 +70,9 @@ static void build_child_join_reltarget(PlannerInfo *root,
RelOptInfo *childrel,
int nappinfos,
AppendRelInfo **appinfos);
+static bool apply_child_basequals(PlannerInfo *root, RelOptInfo *rel,
+ RelOptInfo *childrel,
+ RangeTblEntry *childRTE,
AppendRelInfo *appinfo);
/*
@@ -324,10 +327,162 @@ build_simple_rel(PlannerInfo *root, int relid,
RelOptInfo *parent)
root->qual_security_level = Max(root->qual_security_level,
list_length(rte->securityQuals));
+ /*
+ * Copy the parent's quals to the child, with appropriate substitution
of
+ * variables. If any constant false or NULL clauses turn up, we can
+ * disregard the child as dummy right away.
+ *
+ * We must copy the quals now, because if the child rel happens to be a
+ * partitioned table, expand_partitioned_rtentry (our caller) can
+ * immediate perform partition prunning using the translated quals.
+ */
+ if (parent)
+ {
+ AppendRelInfo *appinfo = root->append_rel_array[relid];
+
+ Assert(appinfo != NULL);
+ if (!apply_child_basequals(root, parent, rel, rte, appinfo))
+ {
+ /*
+ * Some restriction clause reduced to constant FALSE or
NULL after
+ * substitution, so this child need not be scanned.
+ */
+ set_dummy_rel_pathlist(rel);
+ }
+ }
+
return rel;
}
/*
+ * apply_child_basequals
+ * Populate childrel's quals based on rel's quals, translating
them using
+ * appinfo.
+ *
+ * If any of the resulting clauses evaluate to false or NULL, we return false
+ * and don't apply any quals. Caller can mark the relation as a dummy rel in
+ * this case, since it needn't be scanned.
+ *
+ * If any resulting clauses evaluate to true, they're unnecessary and we don't
+ * apply then.
+ */
+static bool
+apply_child_basequals(PlannerInfo *root, RelOptInfo *rel,
+ RelOptInfo *childrel, RangeTblEntry
*childRTE,
+ AppendRelInfo *appinfo)
+{
+ List *childquals;
+ Index cq_min_security;
+ ListCell *lc;
+
+ /*
+ * The child rel's targetlist might contain non-Var expressions, which
+ * means that substitution into the quals could produce opportunities
for
+ * const-simplification, and perhaps even pseudoconstant quals.
Therefore,
+ * transform each RestrictInfo separately to see if it reduces to a
+ * constant or pseudoconstant. (We must process them separately to keep
+ * track of the security level of each qual.)
+ */
+ childquals = NIL;
+ cq_min_security = UINT_MAX;
+ foreach(lc, rel->baserestrictinfo)
+ {
+ RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
+ Node *childqual;
+ ListCell *lc2;
+
+ Assert(IsA(rinfo, RestrictInfo));
+ childqual = adjust_appendrel_attrs(root,
+
(Node *) rinfo->clause,
+
1, &appinfo);
+ childqual = eval_const_expressions(root, childqual);
+ /* check for flat-out constant */
+ if (childqual && IsA(childqual, Const))
+ {
+ if (((Const *) childqual)->constisnull ||
+ !DatumGetBool(((Const *)
childqual)->constvalue))
+ {
+ /* Restriction reduces to constant FALSE or
NULL */
+ return false;
+ }
+ /* Restriction reduces to constant TRUE, so drop it */
+ continue;
+ }
+ /* might have gotten an AND clause, if so flatten it */
+ foreach(lc2, make_ands_implicit((Expr *) childqual))
+ {
+ Node *onecq = (Node *) lfirst(lc2);
+ bool pseudoconstant;
+
+ /* check for pseudoconstant (no Vars or volatile
functions) */
+ pseudoconstant =
+ !contain_vars_of_level(onecq, 0) &&
+ !contain_volatile_functions(onecq);
+ if (pseudoconstant)
+ {
+ /* tell createplan.c to check for gating quals
*/
+ root->hasPseudoConstantQuals = true;
+ }
+ /* reconstitute RestrictInfo with appropriate
properties */
+ childquals = lappend(childquals,
+
make_restrictinfo((Expr *) onecq,
+
rinfo->is_pushed_down,
+
rinfo->outerjoin_delayed,
+
pseudoconstant,
+
rinfo->security_level,
+
NULL, NULL, NULL));
+ /* track minimum security level among child quals */
+ cq_min_security = Min(cq_min_security,
rinfo->security_level);
+ }
+ }
+
+ /*
+ * In addition to the quals inherited from the parent, we might have
+ * securityQuals associated with this particular child node. (Currently
+ * this can only happen in appendrels originating from UNION ALL;
+ * inheritance child tables don't have their own securityQuals, see
+ * expand_inherited_rtentry().) Pull any such securityQuals up into the
+ * baserestrictinfo for the child. This is similar to
+ * process_security_barrier_quals() for the parent rel, except that we
+ * can't make any general deductions from such quals, since they don't
+ * hold for the whole appendrel.
+ */
+ if (childRTE->securityQuals)
+ {
+ Index security_level = 0;
+
+ foreach(lc, childRTE->securityQuals)
+ {
+ List *qualset = (List *) lfirst(lc);
+ ListCell *lc2;
+
+ foreach(lc2, qualset)
+ {
+ Expr *qual = (Expr *) lfirst(lc2);
+
+ /* not likely that we'd see constants here, so
no check */
+ childquals = lappend(childquals,
+
make_restrictinfo(qual,
+
true, false, false,
+
security_level,
+
NULL, NULL, NULL));
+ cq_min_security = Min(cq_min_security,
security_level);
+ }
+ security_level++;
+ }
+ Assert(security_level <= root->qual_security_level);
+ }
+
+ /*
+ * OK, we've got all the baserestrictinfo quals for this child.
+ */
+ childrel->baserestrictinfo = childquals;
+ childrel->baserestrict_min_security = cq_min_security;
+
+ return true;
+}
+
+/*
* add_appendrel_other_rels
* This adds the "other rel" RelOptInfos a given "appendrel"
baserel
*
@@ -340,6 +495,7 @@ add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
ListCell *l;
RelOptInfo *rel;
Relation relation = NULL;
+ PartitionDesc partdesc = NULL;
int i;
/*
@@ -364,6 +520,7 @@ add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
rel->part_rels = (RelOptInfo **)
palloc0(sizeof(RelOptInfo *) * rel->nparts);
relation = table_open(rte->relid, NoLock);
+ partdesc = RelationGetPartitionDesc(relation);
}
/*
@@ -392,16 +549,23 @@ add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
* For partitioned parents, we also need to add childrel to its
* part_rels array. The order in which child tables appear in
* append_rel_list is same as the order in which they appear in
the
- * parent's PartitionDesc, so assigning partitions like this
works.
+ * parent's PartitionDesc. But considering that some
partitions may
+ * have been pruned, we need to look up the correct position to
+ * assign this childrel by scanning the PartitionDesc.
*/
if (rel->part_scheme != NULL)
{
- Assert(rel->nparts > 0 && i < rel->nparts);
- rel->part_rels[i] = childrel;
+ Assert(partdesc != NULL);
+ for (i = 0; i < partdesc->nparts; i++)
+ {
+ if (childrte->relid == partdesc->oids[i])
+ {
+ rel->part_rels[i] = childrel;
+ break;
+ }
+ }
}
- i++;
-
/* Child may itself be an inherited relation. */
if (childrte->inh)
add_appendrel_other_rels(root, childrte, childRTindex);
diff --git a/src/backend/partitioning/partprune.c
b/src/backend/partitioning/partprune.c
index 8c9721935d..30ebce4e40 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -45,6 +45,7 @@
#include "nodes/makefuncs.h"
#include "nodes/nodeFuncs.h"
#include "optimizer/appendinfo.h"
+#include "optimizer/cost.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
#include "partitioning/partprune.h"
@@ -433,17 +434,23 @@ make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
* is, not pruned already).
*/
subplan_map = (int *) palloc(nparts * sizeof(int));
+ memset(subplan_map, -1, nparts * sizeof(int));
subpart_map = (int *) palloc(nparts * sizeof(int));
+ memset(subpart_map, -1, nparts * sizeof(int));
present_parts = NULL;
for (i = 0; i < nparts; i++)
{
RelOptInfo *partrel = subpart->part_rels[i];
- int subplanidx =
relid_subplan_map[partrel->relid] - 1;
- int subpartidx =
relid_subpart_map[partrel->relid] - 1;
+ int subplanidx;
+ int subpartidx;
- subplan_map[i] = subplanidx;
- subpart_map[i] = subpartidx;
+ /* Skip processing pruned partitions. */
+ if (partrel == NULL)
+ continue;
+
+ subplan_map[i] = subplanidx =
relid_subplan_map[partrel->relid] - 1;
+ subpart_map[i] = subpartidx =
relid_subpart_map[partrel->relid] - 1;
if (subplanidx >= 0)
{
present_parts = bms_add_member(present_parts,
i);
@@ -537,23 +544,20 @@ gen_partprune_steps(RelOptInfo *rel, List *clauses, bool
*contradictory)
/*
* prune_append_rel_partitions
- * Returns RT indexes of the minimum set of child partitions which
must
- * be scanned to satisfy rel's baserestrictinfo quals.
+ * Returns indexes into rel->part_rels of the minimum set of child
+ * partitions which must be scanned to satisfy rel's
baserestrictinfo
+ * quals.
*
* Callers must ensure that 'rel' is a partitioned table.
*/
-Relids
+Bitmapset *
prune_append_rel_partitions(RelOptInfo *rel)
{
- Relids result;
List *clauses = rel->baserestrictinfo;
List *pruning_steps;
bool contradictory;
PartitionPruneContext context;
- Bitmapset *partindexes;
- int i;
- Assert(clauses != NIL);
Assert(rel->part_scheme != NULL);
/* If there are no partitions, return the empty set */
@@ -561,6 +565,13 @@ prune_append_rel_partitions(RelOptInfo *rel)
return NULL;
/*
+ * If pruning is disabled or if there are no clauses to prune with,
+ * return all partitions.
+ */
+ if (!enable_partition_pruning || clauses == NIL)
+ return bms_add_range(NULL, 0, rel->nparts - 1);
+
+ /*
* Process clauses. If the clauses are found to be contradictory, we
can
* return the empty set.
*/
@@ -587,15 +598,7 @@ prune_append_rel_partitions(RelOptInfo *rel)
context.evalexecparams = false;
/* Actual pruning happens here. */
- partindexes = get_matching_partitions(&context, pruning_steps);
-
- /* Add selected partitions' RT indexes to result. */
- i = -1;
- result = NULL;
- while ((i = bms_next_member(partindexes, i)) >= 0)
- result = bms_add_member(result, rel->part_rels[i]->relid);
-
- return result;
+ return get_matching_partitions(&context, pruning_steps);
}
/*
--
2.11.0
From 5ba60de97462198a698e427de88893734b195538 Mon Sep 17 00:00:00 2001
From: amit <amitlangot...@gmail.com>
Date: Mon, 4 Mar 2019 11:57:04 +0900
Subject: [PATCH v26 5/6] Teach planner to only process unpruned partitions
This adds a bitmapset field live_parts to RelOptInfo and stores
the partition indexes of non-dummy partitions in it, meaning it
contains indexes of only those partitions that have a non-NULL
RelOptInfo present in its parent's RelOptInfo's part_rels array.
This speeds up processing partitioned table's partitions compared
to going through the whole part_rels array in a number of places.
Partitionwise join still has to visit each member of part_rels,
because pruned children may fall on the nullable side of an outer
join and an empty Result path must be present for building the
outer join paths.
---
src/backend/optimizer/path/joinrels.c | 3 +++
src/backend/optimizer/plan/planner.c | 18 +++++++++---------
src/backend/optimizer/util/inherit.c | 1 +
src/backend/optimizer/util/relnode.c | 4 ++++
src/backend/partitioning/partprune.c | 18 +++++-------------
src/include/nodes/pathnodes.h | 4 ++++
6 files changed, 26 insertions(+), 22 deletions(-)
diff --git a/src/backend/optimizer/path/joinrels.c
b/src/backend/optimizer/path/joinrels.c
index 66a5cf3334..57158d90ec 100644
--- a/src/backend/optimizer/path/joinrels.c
+++ b/src/backend/optimizer/path/joinrels.c
@@ -1470,6 +1470,9 @@ try_partitionwise_join(PlannerInfo *root, RelOptInfo
*rel1, RelOptInfo *rel2,
populate_joinrel_with_paths(root, child_rel1, child_rel2,
child_joinrel, child_sjinfo,
child_restrictlist);
+ if (!IS_DUMMY_REL(child_joinrel))
+ joinrel->live_parts =
bms_add_member(joinrel->live_parts,
+
cnt_parts);
}
if (baserel1)
diff --git a/src/backend/optimizer/plan/planner.c
b/src/backend/optimizer/plan/planner.c
index 7fd1366652..4b6d5b87e5 100644
--- a/src/backend/optimizer/plan/planner.c
+++ b/src/backend/optimizer/plan/planner.c
@@ -7130,7 +7130,9 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
List *live_children = NIL;
/* Adjust each partition. */
- for (partition_idx = 0; partition_idx < rel->nparts;
partition_idx++)
+ partition_idx = -1;
+ while ((partition_idx = bms_next_member(rel->live_parts,
+
partition_idx)) >= 0)
{
RelOptInfo *child_rel = rel->part_rels[partition_idx];
ListCell *lc;
@@ -7138,9 +7140,7 @@ apply_scanjoin_target_to_paths(PlannerInfo *root,
int nappinfos;
List *child_scanjoin_targets = NIL;
- /* Skip processing pruned partitions. */
- if (child_rel == NULL)
- continue;
+ Assert(child_rel != NULL);
/* Translate scan/join targets for this child. */
appinfos = find_appinfos_by_relids(root,
child_rel->relids,
@@ -7221,7 +7221,6 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
PartitionwiseAggregateType patype,
GroupPathExtraData *extra)
{
- int nparts = input_rel->nparts;
int cnt_parts;
List *grouped_live_children = NIL;
List *partially_grouped_live_children = NIL;
@@ -7233,7 +7232,9 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
partially_grouped_rel != NULL);
/* Add paths for partitionwise aggregation/grouping. */
- for (cnt_parts = 0; cnt_parts < nparts; cnt_parts++)
+ cnt_parts = -1;
+ while ((cnt_parts = bms_next_member(input_rel->live_parts,
+
cnt_parts)) >= 0)
{
RelOptInfo *child_input_rel = input_rel->part_rels[cnt_parts];
PathTarget *child_target = copy_pathtarget(target);
@@ -7243,9 +7244,8 @@ create_partitionwise_grouping_paths(PlannerInfo *root,
RelOptInfo *child_grouped_rel;
RelOptInfo *child_partially_grouped_rel;
- /* Skip processing pruned partitions. */
- if (child_input_rel == NULL)
- continue;
+ /* A live partition must have a RelOptInfo. */
+ Assert(child_input_rel != NULL);
/* Input child rel must have a path */
Assert(child_input_rel->pathlist != NIL);
diff --git a/src/backend/optimizer/util/inherit.c
b/src/backend/optimizer/util/inherit.c
index e2c1f341ce..e40ab159a3 100644
--- a/src/backend/optimizer/util/inherit.c
+++ b/src/backend/optimizer/util/inherit.c
@@ -280,6 +280,7 @@ expand_partitioned_rtentry(PlannerInfo *root, RangeTblEntry
*parentrte,
rel = find_base_rel(root, parentRTindex);
if (partdesc->nparts > 0)
live_parts = prune_append_rel_partitions(rel);
+ rel->live_parts = live_parts;
expand_planner_arrays(root, bms_num_members(live_parts) + 1);
Assert(rel->part_rels == NULL);
if (partdesc->nparts > 0)
diff --git a/src/backend/optimizer/util/relnode.c
b/src/backend/optimizer/util/relnode.c
index adc06cf69a..63a6ec2548 100644
--- a/src/backend/optimizer/util/relnode.c
+++ b/src/backend/optimizer/util/relnode.c
@@ -561,6 +561,7 @@ add_appendrel_other_rels(PlannerInfo *root, RangeTblEntry
*rte, Index rti)
if (childrte->relid == partdesc->oids[i])
{
rel->part_rels[i] = childrel;
+ rel->live_parts =
bms_add_member(rel->live_parts, i);
break;
}
}
@@ -1987,6 +1988,9 @@ build_joinrel_partition_info(RelOptInfo *joinrel,
RelOptInfo *outer_rel,
joinrel->partexprs[cnt] = partexpr;
joinrel->nullable_partexprs[cnt] = nullable_partexpr;
}
+
+ /* Partitions will be added by try_partitionwise_join. */
+ joinrel->live_parts = NULL;
}
/*
diff --git a/src/backend/partitioning/partprune.c
b/src/backend/partitioning/partprune.c
index 30ebce4e40..0f990ef6a2 100644
--- a/src/backend/partitioning/partprune.c
+++ b/src/backend/partitioning/partprune.c
@@ -437,29 +437,21 @@ make_partitionedrel_pruneinfo(PlannerInfo *root,
RelOptInfo *parentrel,
memset(subplan_map, -1, nparts * sizeof(int));
subpart_map = (int *) palloc(nparts * sizeof(int));
memset(subpart_map, -1, nparts * sizeof(int));
- present_parts = NULL;
+ present_parts = bms_copy(subpart->live_parts);
- for (i = 0; i < nparts; i++)
+ i = -1;
+ while ((i = bms_next_member(present_parts, i)) >= 0)
{
RelOptInfo *partrel = subpart->part_rels[i];
int subplanidx;
int subpartidx;
- /* Skip processing pruned partitions. */
- if (partrel == NULL)
- continue;
-
+ Assert(partrel != NULL);
subplan_map[i] = subplanidx =
relid_subplan_map[partrel->relid] - 1;
subpart_map[i] = subpartidx =
relid_subpart_map[partrel->relid] - 1;
+ /* Record finding this subplan */
if (subplanidx >= 0)
- {
- present_parts = bms_add_member(present_parts,
i);
-
- /* Record finding this subplan */
subplansfound = bms_add_member(subplansfound,
subplanidx);
- }
- else if (subpartidx >= 0)
- present_parts = bms_add_member(present_parts,
i);
}
pinfo = makeNode(PartitionedRelPruneInfo);
diff --git a/src/include/nodes/pathnodes.h b/src/include/nodes/pathnodes.h
index c2e292e54c..53d0485964 100644
--- a/src/include/nodes/pathnodes.h
+++ b/src/include/nodes/pathnodes.h
@@ -712,6 +712,10 @@ typedef struct RelOptInfo
List *partition_qual; /* partition constraint */
struct RelOptInfo **part_rels; /* Array of RelOptInfos of partitions,
*
stored in the same order of bounds */
+ Bitmapset *live_parts; /* Indexes into part_rels of the
non-NULL
+ * RelOptInfos
of unpruned partitions; exists
+ * to avoid
having to iterate over the entire
+ * part_rels
array to filter NULL entries. */
List **partexprs; /* Non-nullable partition key
expressions. */
List **nullable_partexprs; /* Nullable partition key expressions.
*/
List *partitioned_child_rels; /* List of RT indexes. */
--
2.11.0
From 4a6048ced9bd94aa2f0f544737a0d1acc2b7c806 Mon Sep 17 00:00:00 2001
From: amit <amitlangot...@gmail.com>
Date: Mon, 4 Mar 2019 16:03:49 +0900
Subject: [PATCH v26 6/6] Don't copy PartitionBoundInfo in
set_relation_partition_info
As long as we hold a lock on the table, it shouldn't change, so
seems pointless to copy, which can get really expensive as the
number of partitions grows beyond thousands.
---
src/backend/optimizer/util/plancat.c | 8 +++++---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/src/backend/optimizer/util/plancat.c
b/src/backend/optimizer/util/plancat.c
index 549af9dc6e..b75b761033 100644
--- a/src/backend/optimizer/util/plancat.c
+++ b/src/backend/optimizer/util/plancat.c
@@ -2082,16 +2082,18 @@ set_relation_partition_info(PlannerInfo *root,
RelOptInfo *rel,
Relation relation)
{
PartitionDesc partdesc;
- PartitionKey partkey;
Assert(relation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
partdesc = RelationGetPartitionDesc(relation);
- partkey = RelationGetPartitionKey(relation);
rel->part_scheme = find_partition_scheme(root, relation);
Assert(partdesc != NULL && rel->part_scheme != NULL);
+ /*
+ * Holding onto the relcache pointer should be OK, as it won't change
+ * under us as long as we're holding the lock on the relation.
+ */
if (partdesc->nparts > 0)
- rel->boundinfo = partition_bounds_copy(partdesc->boundinfo,
partkey);
+ rel->boundinfo = partdesc->boundinfo;
rel->nparts = partdesc->nparts;
set_baserel_partition_key_exprs(relation, rel);
rel->partition_qual = RelationGetPartitionQual(relation);
--
2.11.0
