Stamatis Zampetakis created HIVE-28259:
------------------------------------------
Summary: Common table expression detection and rewrites using CBO
Key: HIVE-28259
URL: https://issues.apache.org/jira/browse/HIVE-28259
Project: Hive
Issue Type: New Feature
Components: CBO
Reporter: Stamatis Zampetakis
Assignee: Stamatis Zampetakis
Hive already provides the means to detect and exploit CTEs via the
{{SharedWorkOptimizer}}. The {{SharedWorkOptimizer}} relies on a series of
heuristic transformations of the physical plan ({{Operator}} DAG) that apply
towards the end of the planning process. The optimizer is quite powerful and
offers various properties ({{hive.optimize.shared.work*}}) through which its
behavior can be fine tuned by users but has also a few drawbacks:
* not cost-based
* limited customization
* complex implementation
This ticket aims to leverage CBO for detecting and exploiting common table
expressions (CTE) in queries in an attempt to alleviate some of the
shortcomings of the {{SharedWorkOptimizer}} and open the road for more powerful
transformations.
The initial work focuses on establishing the general design and the main APIs
for CBO based CTE transformations.
The main idea is to model CTEs as materialized views (MVs) and rely on existing
MV rewrite logic to incorporate them in the plan in a cost-based fashion as a
new CBO planning phase.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
