Hello Aman Sinha, Fang-Yu Rao, Riza Suminto, Joe McDonnell, Michael Smith,
Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22425
to look at the new patch set (#13).
Change subject: IMPALA-12959: Calcite planner: Implement count star
optimization...
......................................................................
IMPALA-12959: Calcite planner: Implement count star optimization...
IMPALA-13779: Handle partition key scan optimization
IMPALA-13780: Handle full acid selects
The 3 commits referenced here are somewhat related in that they all
involve changes for the HdfsScanRel column layout and have been
combined.
For the optimizations, some infrastructure code was added. Information
from the Aggregation RelNode is needed by the TableScan RelNode and
vice versa. The mechanism to send information to children RelNodes is
by using the ParentPlanRelContext. The mechanism for sending information
up to the parent is by using the NodeWithExprs object. If the conditions
are met for the optimizations (equivalent to the conditions in the current
Impala planner), the optimizations are applied.
For count star optimization, the STAT_NUM_ROWS fake column is added to hold
the information, and then the aggregate applies a sum_init_zero on this
column.
For partition key scan, if the conditions are met, the Impala HdfsScanNode
is sent a flag in its constructor that handles the optimization.
For acid selects, the SingleNodePlanner has code to handle the additional
PlanNodes needed. Some code involving column number calculation was needed
to deal with the extra columns that are present in a full acid table.
One extra note: In HdfsScanNode, a Preconditions check was removed.
This state check ensured that the countStarSlot only existed when the
aggregation substitution map was set. This does not apply to Calcite which
does not use the substitution map to handle the count star optimization.
Change-Id: I975beefedd2cceb34dad0f93343a46d1b7094c13
---
M fe/src/main/java/org/apache/impala/planner/HdfsScanNode.java
M fe/src/main/java/org/apache/impala/planner/ScanNode.java
M fe/src/main/java/org/apache/impala/planner/SingleNodePlanner.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaAggRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaAnalyticRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaFilterRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaHdfsScanRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaPlanRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaSortRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeWithExprs.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ParentPlanRelContext.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/phys/ImpalaHdfsScanNode.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java
15 files changed, 359 insertions(+), 82 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/22425/13
--
To view, visit http://gerrit.cloudera.org:8080/22425
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I975beefedd2cceb34dad0f93343a46d1b7094c13
Gerrit-Change-Number: 22425
Gerrit-PatchSet: 13
Gerrit-Owner: Steve Carlin <scar...@cloudera.com>
Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <scar...@cloudera.com>
