[
https://issues.apache.org/jira/browse/FLINK-7866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16235869#comment-16235869
]
Till Rohrmann commented on FLINK-7866:
--------------------------------------
[~sihuazhou] after addressing this issue, it might make sense to create a
dedicated Flink improvement proposal for further scheduling changes. I think
Flink's scheduling algorithm will need some revision if we want to properly
include state locality preferences in our scheduling decision.
The first thing to do would be to collect all the requirements we have for a
proper scheduling algorithm. Next thing would be to think about an algorithm
which can calculate a good matching between the different requirements. Ideally
this algorithm works incrementally such that we can interrupt it at any given
point of time if it should take too long.
What do you think?
> Weigh list of preferred locations for scheduling
> ------------------------------------------------
>
> Key: FLINK-7866
> URL: https://issues.apache.org/jira/browse/FLINK-7866
> Project: Flink
> Issue Type: Improvement
> Components: Scheduler
> Affects Versions: 1.4.0, 1.3.2
> Reporter: Till Rohrmann
> Assignee: Sihua Zhou
> Priority: Major
> Fix For: 1.5.0
>
>
> [~sihuazhou] proposed to not only use the list of preferred locations to
> decide where to schedule a task, but to also weigh the list according to how
> often a location appeared and then select the location based on the weight.
> That way, we would obtain better locality in some cases.
> Example:
> Preferred locations list: {{[location1, location2, location2]}}
> Weighted preferred locations list {{[(location2 , 2), (location1, 1)]}}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
