[
https://issues.apache.org/jira/browse/IMPALA-13982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17985961#comment-17985961
]
ASF subversion and git services commented on IMPALA-13982:
----------------------------------------------------------
Commit aba3a705a4c624b2d77e0ab92bf8c37de2d338e0 in impala's branch
refs/heads/master from Fang-Yu Rao
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=aba3a705a ]
IMPALA-13982: Support regular views for Calcite planner in Impala
Before this patch, the Calcite planner in Impala only supported inline
views like 'temp' in the following query.
select id from (
select * from functional.alltypes
) as temp;
Regular views, on the other hand, were not supported. For instance, the
Calcite planner in Impala did not support regular views like
'functional.alltypes_view' created via the following statement and
hence queries against such regular views like
"select id from functional.alltypes_view" were not supported.
CREATE VIEW functional.alltypes_view
AS SELECT * FROM functional.alltypes;
This patch adds the support for regular views to the Calcite planner
via adding a ViewTable for each regular view in the given query
when populating the Calcite schema. This is similar to how regular
views are supported in PlannerTest#testView() at
https://github.com/apache/calcite/blob/main/core/src/test/java/org/apache/calcite/tools/PlannerTest.java
where the regular view to be tested is added in
https://github.com/apache/calcite/blob/main/testkit/src/main/java/org/apache/calcite/test/CalciteAssert.java.
We do not have to use or extend ViewTableMacro in
Apache Calcite because the information about the data types
returned from a regular view is already available in its respective
FeTable. Therefore, there is no need to parse the SQL statement
representing the regular view and collect the metadata of tables
referenced by the regular view as done by ViewTableMacro.
The patch supports the following cases, where
'functional.alltypes_view' is a regular view defined as
"SELECT * FROM functional.alltypes".
1. select id from functional.alltypes_view.
2. select alltypes_view.id from functional.alltypes_view.
3. select functional.alltypes_view.id from functional.alltypes_view.
Joining a regular view with an HDFS table like the following is also
supported.
select alltypestiny.id
from functional.alltypes_view, functional.alltypestiny
Note that after this patch, queries against regular views are supported
only in the legacy catalog mode but not the local catalog mode. In
fact, queries against HDFS tables in the local catalog mode are not
supported yet by the Calcite planner either. We will deal with this in
IMPALA-14080.
Testing:
- Added test cases mentioned above to calcite.test. This makes sure
the test cases are supported when we start the Impala server with
the flag of '--use_calcite_planner=true'.
- Manually verified the test cases above are supported if we start
the Impala server with the environment variable USE_CALCITE_PLANNER
set to true and the query option use_calcite_planner set to 1.
Change-Id: I600aae816727ae942fb221fae84c2aac63ae1893
Reviewed-on: http://gerrit.cloudera.org:8080/22883
Tested-by: Impala Public Jenkins <[email protected]>
Reviewed-by: Joe McDonnell <[email protected]>
> Calcite Planner: Support views for the Calcite planner in Impala
> ----------------------------------------------------------------
>
> Key: IMPALA-13982
> URL: https://issues.apache.org/jira/browse/IMPALA-13982
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Fang-Yu Rao
> Assignee: Fang-Yu Rao
> Priority: Major
>
> We should be able to support views for the Calcite planner in Impala without
> a lot of code changes to the current implementation.
> The ideas is to add a {{TableVisitor}} to traverse the AST right before we
> analyze the AST using Calcite's own {{SqlValidator}} in
> [https://github.com/apache/impala/blob/master/java/calcite-planner/src/main/java/org/apache/impala/calcite/service/CalciteAnalysisDriver.java]
> and replace nodes representing views with the underlying definitions of
> views.
> We also have to take authorization into consideration. Recall that for a view
> created by a non-superuser, Impala further requires the requesting user to
> have the privileges on the underlying tables referenced by the view. Refer to
> [IMPALA-10122 (Part 2): Allow accessing views created by
> non-superusers|https://github.com/apache/impala/commit/607dc1f170fb8cc013d8ec33fccee59ac4352fe4]
> for more details. This implies that given an AST after the node replacement
> mentioned above, we need to distinguish a {{SqlSelect}} node derived from a
> view from a {{SqlSelect}} that was not derived from a view.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
