[
https://issues.apache.org/jira/browse/FLINK-31079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lijie Wang closed FLINK-31079.
------------------------------
Resolution: Done
> Release Testing: Verify FLINK-29663 Further improvements of adaptive batch
> scheduler
> ------------------------------------------------------------------------------------
>
> Key: FLINK-31079
> URL: https://issues.apache.org/jira/browse/FLINK-31079
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination
> Reporter: Lijie Wang
> Assignee: miamiaoxyz
> Priority: Blocker
> Fix For: 1.17.0
>
> Attachments: image-2023-02-22-14-00-13-646.png
>
>
> This task aims to verify FLINK-29663 which improves the adaptive batch
> scheduler.
> Before the change of FLINK-29663, adaptive batch scheduler will distribute
> subpartitoins according to the number of subpartitions, make different
> downstream subtasks consume roughly the same number of subpartitions. This
> will lead to imbalance loads of different downstream tasks when the
> subpartitions contain different amounts of data.
> To solve this problem, in FLINK-29663, we let the adaptive batch scheduler
> distribute subpartitoins according to the amount of data, so that different
> downstream subtasks consume roughly the same amount of data. Note that
> currently it only takes effect for All-To-All edges.
> The documentation of adaptive scheduler can be found
> [here|https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/elastic_scaling/#adaptive-batch-scheduler]
> One can verify it by creating intended data skew on All-To-All edges.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
