[
https://issues.apache.org/jira/browse/FLINK-28981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17599221#comment-17599221
]
Yunhong Zheng commented on FLINK-28981:
---------------------------------------
Hi, [~hxb] [~zhuzh] [~godfreyhe] . I have finished tests on InputFormat
sources. I modified datagen source‘s code. First, I let some tasks sleep when
they process each records according to their subtask ID . and then gradually
reduce the sleep time of these task according to the number of attempts. In
this way, the functions mentioned in PR can be simulated, and the verification
results are in line with expectations.
For me, I will continue to test FLIP-27 source if time is enough.
> Release Testing: Verify FLIP-245 sources speculative execution
> --------------------------------------------------------------
>
> Key: FLINK-28981
> URL: https://issues.apache.org/jira/browse/FLINK-28981
> Project: Flink
> Issue Type: Sub-task
> Components: Connectors / Common, Runtime / Coordination
> Reporter: Zhu Zhu
> Assignee: Yunhong Zheng
> Priority: Blocker
> Labels: release-testing
> Fix For: 1.16.0
>
>
> Speculative execution is introduced in Flink 1.16 to deal with temporary slow
> tasks caused by slow nodes. This feature currently consists of 4 FLIPs:
> - FLIP-168: Speculative Execution core part
> - FLIP-224: Blocklist Mechanism
> - FLIP-245: Source Supports Speculative Execution
> - FLIP-249: Flink Web UI Enhancement for Speculative Execution
> This ticket aims for verifying FLIP-245, along with FLIP-168, FLIP-224 and
> FLIP-249.
> More details about this feature and how to use it can be found in this
> [document|https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/speculative_execution/].
> To do the verification, the process can be:
> - Write Flink jobs which has some {{source}} subtasks running much slower
> than others. 3 kinds of sources should be verified, including
> -- [Source
> functions|https://github.com/apache/flink/blob/master/flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/SourceFunction.java]
> -- [InputFormat
> sources|https://github.com/apache/flink/blob/master/flink-core/src/main/java/org/apache/flink/api/common/io/InputFormat.java]
> -- [FLIP-27 new
> sources|https://github.com/apache/flink/blob/master//flink-core/src/main/java/org/apache/flink/api/connector/source/Source.java]
> - Modify Flink configuration file to enable speculative execution and tune
> the configuration as you like
> - Submit the job. Checking the web UI, logs, metrics and produced result.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
