[
https://issues.apache.org/jira/browse/FLINK-16165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18112954#comment-18112954
]
Bonnie Varghese edited comment on FLINK-16165 at 9/8/26 6:31 PM:
-----------------------------------------------------------------
Sometimes, even if we provide statistics (eg: only row numbers but no column
stats i.e. ndv), the planner can incorrectly pick a nested loop join.
We could make the NestedLoopJoin the last priority during planning time itself.
One way could be to always fallback to HashJoin if there is an equi join
condition available. Only use NestedLoopJoin if a join hint is provided or if
there is no equi join condition provided.
I have tested this locally and it works well with the tpc-ds dataset.
was (Author: JIRAUSER300319):
Sometimes, even if we provide statistics (eg: only row numbers but no column
numbers), the planner can incorrectly pick a nested loop join.
We could make the NestedLoopJoin the last priority during planning time itself.
One way could be to always fallback to HashJoin if there is an equi join
condition available. Only use NestedLoopJoin if a join hint is provided or if
there is no equi join condition provided.
I have tested this locally and it works well with the tpc-ds dataset.
> NestedLoopJoin fallback to HashJoin when build records number is very large
> ---------------------------------------------------------------------------
>
> Key: FLINK-16165
> URL: https://issues.apache.org/jira/browse/FLINK-16165
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Runtime
> Reporter: Jingsong Lee
> Priority: Not a Priority
> Labels: auto-deprioritized-critical, auto-deprioritized-major,
> auto-deprioritized-minor
>
> Now, if the statistic is not so accurate, maybe choose wrong join type to
> nested loop join.
> If build records number is very large, this lead to very slow join, the user
> looks like the join is in Hang.
> It is a stability problem, We should fallback to hash join in runtime to
> avoid this hang.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
