[
https://issues.apache.org/jira/browse/CALCITE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17866756#comment-17866756
]
ruanhui commented on CALCITE-6363:
----------------------------------
Hi all
I tried to re-implement predicate derivation in a new commit. Compared with a
formal PR, it is more like a POC now. Looking forward to your suggestions.
*Why this change was been made*
As I commented above, predicate inference is not available in VolcanoPlanner
currently, so this PR tries to propose a temporary solution to help implement
predicate inference.
*Main changes*
1. Introduce a new method in RelOptUtil to implement predicate inference, which
basically copies RelMdPredicates.JoinConditionBasedPredicateInference, but
makes some simplifications as follows:
1.1 Because the predicate is passed directly from the join, rather than pulled
up from the filter node after the predicate is pushed down, there is no need to
convert RexInputRef
1.2 FilterJoinRule will classify the generated predicates, so the new method
does not need to classify the predicates, and just returns the generated
predicates directly
2. Call this method in FilterJoinRule, just after simplifying Outer Joins
3. Introduced a new configuration item in FilterJoinRule to determine whether
to enable predicate inference, and turned it off by default
4. Added some unit tests
*Impact*
1. Predicate inference is turned off by default, so it has no impact on the
current logic
2. Applicable to Inner Join, Outer Join and Semi-Anti Join
> Introduce a rule to derive more filters from inner join condition
> -----------------------------------------------------------------
>
> Key: CALCITE-6363
> URL: https://issues.apache.org/jira/browse/CALCITE-6363
> Project: Calcite
> Issue Type: New Feature
> Components: core
> Reporter: ruanhui
> Priority: Minor
> Labels: pull-request-available
>
> Sometimes we can infer more predicates from inner Join , for example, in the
> query
> SELECT * FROM ta INNER JOIN tb ON ta.x = tb.y WHERE ta.x > 10
> we can infer condition tb.y > 10 and we can push it down to the table tb.
> In this way, it is possible to reduce the amount of data involved in the Join.
> To achieve this, here is my idea.
> The core data strucature is two Multimap:
> predicateMap : a map for inputRef to corresponding predicate such as: $1 ->
> [$1 > 10, $1 < 20, $1 = $2]
> equivalenceMap : a map for inputRef to corresponding equivalent values or
> inputRefs such as: $1 -> [$2, 1]
> The filter derivation is divided into 4 steps:
> 1. construct predicate map and equivalence map by traversing all conjunctions
> in the condition
> 2. search map and rewrite predicates with equivalent inputRefs or literals
> 2.1 find all inputRefs that are equivalent to the current inputRef, and then
> rewrite all predicates involving equivalent inputRefs using inputRef, for
> example if we have inputRef $1 = equivInputRef $2, then we can rewrite \{$2 =
> 10} to \{$1 = 10}.
> 2.2 find all predicates involving current inputRef. If any predicate refers
> to another inputRef, rewrite the predicate with the literal/constant
> equivalent to that inputRef, such as: if we have inputRef \{$1 > $2} and \{$2
> = 10} then we can infer new condition \{$1 > 10}.
> 2.3 derive new predicates based on equivalence relation in equivalenceMultimap
> 3. compose all original predicates and derived predicates
> 4. simplify expression such as range merging, like \{$1 > 10 AND $1 > 20} =>
> \{$1 > 20}, \{$1 > $2 AND $1 > $2} => \{$1 > $2}
> Anyone interested in this, please feel free to comment on this issue.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
