[
https://issues.apache.org/jira/browse/FLINK-37160?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junrui Lee updated FLINK-37160:
-------------------------------
Description:
In Flink 2.0, we support the capability of adaptive broadcast join for batch
jobs, which can determine at runtime whether the small table can be broadcast
based on the actual input data size of the join operator. It then transforms
the operators that meet the condition to achieve better join performance.
We may need the following tests:
1. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` is
set to 'auto', we may need to construct a case (e.g. a case where the source
includes a downstream filter operator) such that the join operator is not
optimized as a broadcast hash join during the table compilation phase but can
be dynamically optimized as a broadcast hash join at runtime.
2. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` is
set to 'runtime_only', to verify that the join operator is not optimized into a
broadcast hash join during the table compilation phase, even if the small table
side meets the broadcast threshold, while also being able to dynamically
convert into a broadcast hash join at runtime.
3. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` as
none, and verify that the join operator will not be optimized into an adaptive
join operator under any circumstances.
was:
This is for cross-team verification of the release 2.0 work item: "Adaptive
broadcast join for batch jobs"
Please complete the following steps before Jan 31.
# Replace the description of the ticket with instructions for the testers.
** If the work item does not need cross-team verification, please comment that
explicitly and close the ticket.
# Un-assign yourself after adding the instructions.
# Either find a cross-team tester who is willing to help and assign to the
ticket, or reach out to the release manager [~xtsong].
Thanks for your contributions. For any questions, feel free to reach out to the
release manager [~xtsong].
> Cross-team verification for "Adaptive broadcast join for batch jobs"
> --------------------------------------------------------------------
>
> Key: FLINK-37160
> URL: https://issues.apache.org/jira/browse/FLINK-37160
> Project: Flink
> Issue Type: Sub-task
> Reporter: Junrui Lee
> Assignee: xingbe
> Priority: Blocker
> Fix For: 2.0.0
>
>
> In Flink 2.0, we support the capability of adaptive broadcast join for batch
> jobs, which can determine at runtime whether the small table can be broadcast
> based on the actual input data size of the join operator. It then transforms
> the operators that meet the condition to achieve better join performance.
> We may need the following tests:
> 1. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` is
> set to 'auto', we may need to construct a case (e.g. a case where the source
> includes a downstream filter operator) such that the join operator is not
> optimized as a broadcast hash join during the table compilation phase but can
> be dynamically optimized as a broadcast hash join at runtime.
> 2. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` is
> set to 'runtime_only', to verify that the join operator is not optimized into
> a broadcast hash join during the table compilation phase, even if the small
> table side meets the broadcast threshold, while also being able to
> dynamically convert into a broadcast hash join at runtime.
> 3. Test the case where `table.optimizer.adaptive-broadcast-join.strategy` as
> none, and verify that the join operator will not be optimized into an
> adaptive join operator under any circumstances.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
