[
https://issues.apache.org/jira/browse/FLINK-14059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16938409#comment-16938409
]
Zhu Zhu commented on FLINK-14059:
---------------------------------
[~xintongsong][~trohrmann] I think the name {{allSourcesInSamePipelinedRegion}}
can be deceiving. Actually we should not put sources into the same logical
region[1] for a streaming job. Otherwise we will only have single-region
streaming jobs, which is not friendly for fine grained recovery.
What we really want is to put streaming job vertices into the same shared
group, even if they are not in the same logical region, so that slots not fewer
than the max parallelism would suffice.
And this is a default behavior which can be overridden by user settings.
So maybe {{allVerticesInSameSlotSharingGroupByDefault}} is better?
When it's true, all logical regions should use the same groupID, otherwise they
should generate their own groupIDs.
[1] When mentioning region, some times we refer to the regions in JobGraph,
sometimes the regions in execution level. To avoid confusion, I'd like to say
"logical region" for JogGraph regions, and "execution region" (or simply
"region") for execution level regions.
> Introduce option allSourcesInSamePipelinedRegion in ExecutionConfig
> -------------------------------------------------------------------
>
> Key: FLINK-14059
> URL: https://issues.apache.org/jira/browse/FLINK-14059
> Project: Flink
> Issue Type: Sub-task
> Components: Runtime / Coordination, Table SQL / Runtime
> Affects Versions: 1.10.0
> Reporter: Xintong Song
> Assignee: Zhu Zhu
> Priority: Major
> Fix For: 1.10.0
>
>
> * Introduce option {{allSourcesInSamePipelinedRegion}} in {{ExecutionConfig}}
> * Set it to {{true}} by default
> * Set it to {{false}} for SQL/Table API bounded batch jobs by the Blink
> planner
> This step should not introduce any behavior changes.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
