Hello Aman Sinha, Joe McDonnell, Csaba Ringhofer, Michael Smith, Impala Public
Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/21239
to look at the new patch set (#13).
Change subject: IMPALA-13043: Implement Join Capability to the Calcite Planner
......................................................................
IMPALA-13043: Implement Join Capability to the Calcite Planner
This commit adds the ability to handle joins in the Calcite planner.
Some of the items worth noting:
There is extra handling in the ImpalaJoinRel class to deal with outer
joins. The AnalyzedTupleIsNullExpr object is needed for processing
which derives from TupleIsNullExpr. Normally, expressions are created
in the CreateExprVisitor, but the join requires that the TupleIsNullExpr
object is wrapped around the expressions retrieved from the inputs.
The execution engine requires separation of the equijoin conditions and
the non-equijoin conditions. Furthermore, the equijoin conditions are
BinaryCompPredicates instead of normal FunctionCallExprs, so the
AnalyzedBinaryCompExpr class had to be created.
Special processing needed to be coded for runtime filter generators.
The conditions needed to be added to the value transfer graph in order
to enable the Impala planner logic to create and push these generators.
There is some special case logic in the join that requires the execution
engine to run on a single node. The original logic can be found in the
JoinNode planner object, but this code isn't executed. The new mechanism
for checking single node executions is passed back to the root via the
NodeWithExprs object, and is checked in the ImpalaJoinRel.useSingleNode
method.
The join also required some rules to be added to the optimizer. If a join
is done through the "ON" clause, Calcite is able to place the join
condition directly in the Join RelNode. However, if it is in the "WHERE"
clause, Calcite creates a Filter RelNode and creates a cross join RelNode
object. Therefore, in order to handle "WHERE" joins, we need to implement
the rules in the optimizer.
Change-Id: I5db097577907d79877f52feff2922000af074ecd
---
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedBinaryCompExpr.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/AnalyzedFunctionCallExpr.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/FunctionResolver.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexCallConverter.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/functions/RexLiteralConverter.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ConvertToImpalaRelRules.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaAggRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaHdfsScanRel.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaJoinRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaPlanRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaProjectRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaSortRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/ImpalaValuesRel.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeCreationUtils.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/node/NodeWithExprs.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/phys/ImpalaHashJoinNode.java
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/rel/phys/ImpalaNestedLoopJoinNode.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteOptimizer.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/ExecRequestCreator.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeConverter.java
M testdata/workloads/functional-query/queries/QueryTest/calcite.test
21 files changed, 941 insertions(+), 52 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/39/21239/13
--
To view, visit http://gerrit.cloudera.org:8080/21239
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I5db097577907d79877f52feff2922000af074ecd
Gerrit-Change-Number: 21239
Gerrit-PatchSet: 13
Gerrit-Owner: Steve Carlin <[email protected]>
Gerrit-Reviewer: Aman Sinha <[email protected]>
Gerrit-Reviewer: Csaba Ringhofer <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Joe McDonnell <[email protected]>
Gerrit-Reviewer: Michael Smith <[email protected]>
