[
https://issues.apache.org/jira/browse/FLINK-30213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17638774#comment-17638774
]
Gyula Fora commented on FLINK-30213:
------------------------------------
The same issue already existed Flink with the reactive scaling logic.
Take the following pipeline as an example:
{code:java}
public class RescaleTest {
public static void main(String[] args) throws Exception {
var env = StreamExecutionEnvironment.getExecutionEnvironment();
var source = env.fromSequence(0, 100000000L).setParallelism(1);
source.getTransformation().setMaxParallelism(1);
source.disableChaining()
.filter(
val -> {
Thread.sleep(1000);
return true;
})
.print();
env.execute();
}
} {code}
If you start this on a cluster with 1 taskslot (parallelism = 1), that will
create forward partitioning between the source and the filter.
Scaling it up by adding one more taskmanager will increase the parallelism of
the filter but not the source (due to the forced max parallelism 1)
leading to the same problem:
!image-2022-11-25-21-58-48-968.png!
Furthermore the UI reports incorrect parallelism numbers but thats just
cosmetics :)
> The edge is wrong when the vertex parallelism is changed
> --------------------------------------------------------
>
> Key: FLINK-30213
> URL: https://issues.apache.org/jira/browse/FLINK-30213
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Configuration
> Affects Versions: 1.17.0
> Reporter: Rui Fan
> Assignee: Maximilian Michels
> Priority: Major
> Fix For: 1.17.0
>
> Attachments: image-2022-11-25-17-02-12-492.png,
> image-2022-11-25-17-06-53-466.png, image-2022-11-25-21-58-48-968.png
>
>
> After FLINK-29501, flink allows overriding JobVertex parallelisms during job
> submission.
> However, the edge should be changed as well. For example, the job has 4
> vertex, and all shipStrategyName of all tasks are forward.
> After the parallelism of the third task is changed to 1, the second and third
> edge should be changed from forward to rebalance. But they are still forward.
>
> And from the second picture, the subtask_1 of sink cannot receive any data.
>
> !image-2022-11-25-17-02-12-492.png|width=1318,height=356!
> !image-2022-11-25-17-06-53-466.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
