Hello Aman Sinha, Riza Suminto, Joe McDonnell, Michael Smith, Steve Carlin,
Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22883
to look at the new patch set (#9).
Change subject: IMPALA-13982: Support views for Calcite planner in Impala
......................................................................
IMPALA-13982: Support views for Calcite planner in Impala
This patch adds the support for regular views to the Calcite planner
via adding a ViewTableMacro for each regular view in the given query
when populating the Calcite schema. This is also how regular views are
supported in PlannerTest#testView() at
https://github.com/apache/calcite/blob/main/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
in Apache Calcite. Specifically, each regular view is added in
CalciteAssert#addSchema() via instantiating a ViewTableMacro so that it
could be expanded and validated later on.
To be able to expand the view in ViewTableMacro, we need a newer
version of protobuf-java, e.g., 3.21.12. Currently 2.5.0 is
transitively pulled in by dependencies like
org.apache.hadoop:hadoop-hdfs. To address this issue, based on the
suggestion from Stamatis Zampetakis, we exclude avatica-core from the
artifacts that are pulled in by calcite-core, and explicitly add
avatica as a dependency, which already shades the protobuf-java it
requires to expand a view.
The patch supports the following cases, where functional.alltypes_view
is a regular view defined as "SELECT * FROM functional.alltypes".
1. select id from functional.alltypes_view.
2. select alltypes_view.id from functional.alltypes_view.
3. select functional.alltypes_view.id from functional.alltypes_view.
Joining a regular view with an HDFS table like the following is also
supported.
select alltypestiny.id
from functional.alltypes_view, functional.alltypestiny
Testing:
- Added test cases mentioned above to calcite.test. This makes sure
the test cases are supported when we start the Impala server with
the flag of '--use_calcite_planner=true'.
- Manually verified the test cases above are supported if we start
the Impala server with the environment variable USE_CALCITE_PLANNER
set to true and the query option use_calcite_planner set to 1.
Change-Id: I600aae816727ae942fb221fae84c2aac63ae1893
---
M java/calcite-planner/pom.xml
A
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/ImpalaViewTableMacro.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java
M testdata/workloads/functional-query/queries/QueryTest/calcite.test
6 files changed, 273 insertions(+), 17 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/22883/9
--
To view, visit http://gerrit.cloudera.org:8080/22883
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I600aae816727ae942fb221fae84c2aac63ae1893
Gerrit-Change-Number: 22883
Gerrit-PatchSet: 9
Gerrit-Owner: Fang-Yu Rao <fangyu....@cloudera.com>
Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <scar...@cloudera.com>
