[
https://issues.apache.org/jira/browse/FLINK-30707?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zhu Zhu closed FLINK-30707.
---------------------------
Release Note: The slow task detecting is improved for speculative
execution. Previously, it only considers the execution time of tasks when
deciding which tasks are slow. It now takes the input data volume of tasks into
account. Tasks which has a longer execution time but consumes more data may not
be considered as slow. This improvement helps to eliminate the negative impacts
of data skew on slow task detecting.
Resolution: Done
> Let speculative execution take input data amount into account when detecting
> slow tasks
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-30707
> URL: https://issues.apache.org/jira/browse/FLINK-30707
> Project: Flink
> Issue Type: Improvement
> Components: Runtime / Coordination
> Affects Versions: 1.17.0
> Reporter: xingbe
> Assignee: xingbe
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.17.0
>
>
> The default slowTaskDetector in speculative execution is based on execution
> time now, if the data amount allocated to each subtask is uneven, time-based
> slow task detection may not reflect the actual performance of the subtask.
> Therefore, it is necessary to take the input information into account when
> detecting slow tasks.
> The major changes are:
> * Need to know the amount of data processed by each execution vertex.
> * ExecutionTimeBasedSlowTaskDetector needs to introduce the amount of input
> data to weight it with execution time.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
