[
https://issues.apache.org/jira/browse/FLINK-12371?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
godfrey he updated FLINK-12371:
-------------------------------
Description:
This issue aims to convert IN/EXISTS to semi-join, and NOT IN/NOT EXISTS to
anti-join.
In Calcite,
[SemiJoin|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/SemiJoin.java]
only represents semi-join, (could not represent anti-join) and requires equi
join condition. Queries like `select * from left where left.a1 in (select
right.a2 from right where left.b1 > right.b2)` and `select * from left where
not exists (select * from right)` could not be converted to Calcite SemiJoin
operator.
To solve the above problem, We need copy the {{SemiJoin}} class to Flink, and
make the following changes:
1. make {{SemiJoin}} extending from {{Join}}, not from {{EquiJoin}}. (to
support non-equi join condition)
2. add {{isAnti}} field attribute to represent anti-join.
Currently, there are no rules to convert (NOT) IN/ (NOT) EXISTS to SemiJoin, so
we need a whole new rule (named {{FlinkSubQueryRemoveRule}}) to meet our
requirement.
was:
This issue aims to convert IN/EXISTS to semi-join, and NOT IN/NOT EXISTS to
anti-join.
In Calcite,
[SemiJoin|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/SemiJoin.java]
only represents semi-join, (could not represent anti-join) and requires equi
join condition. Queries like `select * from left where left.a1 in (select
right.a2 from right where left.b1 > right.b2)` and `select * from left where
not exists (select * from right)` could not be converted to Calcite SemiJoin
operator.
To solve the above problem, We need copy the {{SemiJoin}} class to Flink, and
make the following changes:
1. make {{SemiJoin}} extending from {{Join}}, not from {{EquiJoin}}. (to
support non-equi join condition)
2. add {{isAnti}} field attribute to represent anti-join.
Currently, there are no rules to convert (NOT) IN/ (NOT) EXISTS to SemiJoin, so
we need a whole new set of rules to meet our requirement.
1. {{FlinkSubQueryRemoveRule}}, a planner rule that converts IN and EXISTS into
semi-join, converts NOT IN and NOT EXISTS into anti-join
2. {{FlinkRewriteSubQueryRule}}, a planner rule that converts filter condition
like: {{(select count(*) from T) > 0}} to {{exists(select * from T)}}
> Add support for converting (NOT) IN/ (NOT) EXISTS to SemiJoin, and generating
> optimized logical plan
> ----------------------------------------------------------------------------------------------------
>
> Key: FLINK-12371
> URL: https://issues.apache.org/jira/browse/FLINK-12371
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Planner
> Reporter: godfrey he
> Assignee: godfrey he
> Priority: Major
>
> This issue aims to convert IN/EXISTS to semi-join, and NOT IN/NOT EXISTS to
> anti-join.
> In Calcite,
> [SemiJoin|https://github.com/apache/calcite/blob/master/core/src/main/java/org/apache/calcite/rel/core/SemiJoin.java]
> only represents semi-join, (could not represent anti-join) and requires equi
> join condition. Queries like `select * from left where left.a1 in (select
> right.a2 from right where left.b1 > right.b2)` and `select * from left where
> not exists (select * from right)` could not be converted to Calcite SemiJoin
> operator.
> To solve the above problem, We need copy the {{SemiJoin}} class to Flink, and
> make the following changes:
> 1. make {{SemiJoin}} extending from {{Join}}, not from {{EquiJoin}}. (to
> support non-equi join condition)
> 2. add {{isAnti}} field attribute to represent anti-join.
> Currently, there are no rules to convert (NOT) IN/ (NOT) EXISTS to SemiJoin,
> so we need a whole new rule (named {{FlinkSubQueryRemoveRule}}) to meet our
> requirement.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
