zhuzhurk commented on a change in pull request #10007: [FLINK-14060][runtime]
Set slot sharing groups according to logical pipelined regions
URL: https://github.com/apache/flink/pull/10007#discussion_r342108224
##########
File path:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/graph/StreamingJobGraphGenerator.java
##########
@@ -596,26 +600,72 @@ public static boolean isChainable(StreamEdge edge,
StreamGraph streamGraph) {
}
private void setSlotSharingAndCoLocation() {
- final HashMap<String, SlotSharingGroup> slotSharingGroups = new
HashMap<>();
- final HashMap<String, Tuple2<SlotSharingGroup,
CoLocationGroup>> coLocationGroups = new HashMap<>();
+ setSlotSharing();
+ setCoLocation();
+ }
+
+ private void setSlotSharing() {
+ final Map<String, SlotSharingGroup> specifiedSlotSharingGroups
= new HashMap<>();
+ final Map<JobVertexID, SlotSharingGroup>
vertexRegionSlotSharingGroups = buildVertexRegionSlotSharingGroups();
for (Entry<Integer, JobVertex> entry : jobVertices.entrySet()) {
- final StreamNode node =
streamGraph.getStreamNode(entry.getKey());
final JobVertex vertex = entry.getValue();
+ final String slotSharingGroupKey =
streamGraph.getStreamNode(entry.getKey()).getSlotSharingGroup();
- // configure slot sharing group
- final String slotSharingGroupKey =
node.getSlotSharingGroup();
final SlotSharingGroup sharingGroup;
+ if (slotSharingGroupKey == null) {
+ sharingGroup = null;
Review comment:
`null` means no slot sharing so that the task will directly allocate a
single slot in `SchedulerImpl`.
However, looks to me there is no chance for a `slotSharingGroupKey` to be
finally null in a `StreamNode`, because it will always be set to a non-null
value returned by `StreamGraphGenerator#determineSlotSharingGroup` in ctor or
later(for iteration head/tail).
So may later we can even drop the non-null assumptions of
slotSharingGroupKey/slotSharingGroup/slotSharingGroupId all along and drop the
related logics, which means we will always allocate logical slot from a shared
slot.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
