Hello Aman Sinha, Riza Suminto, Joe McDonnell, Michael Smith, Steve Carlin,
Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/22883
to look at the new patch set (#13).
Change subject: IMPALA-13982: Support regular views for Calcite planner in
Impala
......................................................................
IMPALA-13982: Support regular views for Calcite planner in Impala
Before this patch, the Calcite planner in Impala only supported inline
views like 'temp' in the following query.
select id from (
select * from functional.alltypes
) as temp;
Regular views, on the other hand, were not supported. For instance, the
Calcite planner in Impala did not support regular views like
'functional.alltypes_view' created via the following statement and
hence queries against such regular views like
"select id from functional.alltypes_view" were not supported.
CREATE VIEW functional.alltypes_view
AS SELECT * FROM functional.alltypes;
This patch adds the support for regular views to the Calcite planner
via adding a ViewTable for each regular view in the given query
when populating the Calcite schema. This is similar to how regular
views are supported in PlannerTest#testView() at
https://github.com/apache/calcite/blob/main/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
where the regular view to be tested is added in
https://github.com/apache/calcite/blob/main/testkit/src/main/java/org/apache/calcite/test/CalciteAssert.java.
We do not have to use or extend ViewTableMacro in
Apache Calcite because the information about the data types
returned from a regular view is already available in its respective
FeTable. Therefore, there is no need to parse the SQL statement
representing the regular view and collect the metadata of tables
referenced by the regular view as done by ViewTableMacro.
The patch supports the following cases, where
'functional.alltypes_view' is a regular view defined as
"SELECT * FROM functional.alltypes".
1. select id from functional.alltypes_view.
2. select alltypes_view.id from functional.alltypes_view.
3. select functional.alltypes_view.id from functional.alltypes_view.
Joining a regular view with an HDFS table like the following is also
supported.
select alltypestiny.id
from functional.alltypes_view, functional.alltypestiny
Note that after this patch, queries against regular views are supported
only in the legacy catalog mode but not the local catalog mode. In
fact, queries against HDFS tables in the local catalog mode are not
supported yet by the Calcite planner either. We will deal with this in
IMPALA-14080.
Testing:
- Added test cases mentioned above to calcite.test. This makes sure
the test cases are supported when we start the Impala server with
the flag of '--use_calcite_planner=true'.
- Manually verified the test cases above are supported if we start
the Impala server with the environment variable USE_CALCITE_PLANNER
set to true and the query option use_calcite_planner set to 1.
Change-Id: I600aae816727ae942fb221fae84c2aac63ae1893
---
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteDb.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/schema/CalciteTable.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteMetadataHandler.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteRelNodeConverter.java
M
java/calcite-planner/src/main/java/org/apache/impala/calcite/type/ImpalaTypeSystemImpl.java
M testdata/workloads/functional-query/queries/QueryTest/calcite.test
7 files changed, 193 insertions(+), 21 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/22883/13
--
To view, visit http://gerrit.cloudera.org:8080/22883
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I600aae816727ae942fb221fae84c2aac63ae1893
Gerrit-Change-Number: 22883
Gerrit-PatchSet: 13
Gerrit-Owner: Fang-Yu Rao <fangyu....@cloudera.com>
Gerrit-Reviewer: Aman Sinha <amsi...@cloudera.com>
Gerrit-Reviewer: Fang-Yu Rao <fangyu....@cloudera.com>
Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com>
Gerrit-Reviewer: Joe McDonnell <joemcdonn...@cloudera.com>
Gerrit-Reviewer: Michael Smith <michael.sm...@cloudera.com>
Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com>
Gerrit-Reviewer: Steve Carlin <scar...@cloudera.com>
