Re: [DISCUSS] decentralized scheduling strategy is needed

2020-01-22 Thread Till Rohrmann
Thanks for reporting the issue HuWeihua. Choosing the right scheduling strategy when using Yarn with potentially infinite resources can be quite hard because you don't know over how many TaskExecutors one should distribute the tasks. It becomes easier if one can configure the minimum number of Task

Re: [DISCUSS] decentralized scheduling strategy is needed

2020-01-15 Thread HuWeihua
Hi, Andrey Thanks for your response. I have checked this Jira ticket and I think it can work in standalone mode which TaskManager has been started before scheduling tasks. But we are currently running flink on yarn in per-job cluster mode. I noticed that this issue has already been raised. I wi

Re: [DISCUSS] decentralized scheduling strategy is needed

2020-01-15 Thread Chesnay Schepler
This is a known issue that's will be fixed in 1.9.2/1.10.0; see https://issues.apache.org/jira/browse/FLINK-12122 . On 15/01/2020 10:07, HuWeihua wrote: Hi, All We encountered some problems during the upgrade from Flink 1.5 to Flink 1.9. Flink's scheduling strategy has changed. Flink 1.9 prefe