Hi Qi Kang, If you means to spread out all taskmanager evenly across the yarn cluster, it is a pity that flink could do nothing. Each per-job flink cluster is an individual application on the yarn cluster, they do not know the existence of others.
Could share the yarn version? If it is above hadoop-3.x, then you should set the *yarn.scheduler.capacity.per-node-heartbeat.multiple-assignments-enabled=false* to avoid assign multiple containers to one nodemanager in a hearbeat. Best, Yang Qi Kang <miraisen...@126.com> 于2019年8月26日周一 下午4:52写道： > Hi, > > > We got 3 Flink jobs running on a 10-node YARN cluster. The jobs were > submitted in a per-job flavor, with same parallelism (10) and number of > slots per TM (2). > > We originally assumed that TMs should automatically spread across the > cluster, but what came out was just the opposite: All 5 TMs from one job > simply went into one single node, thus leaving 7 nodes (almost) idle, and 3 > nodes under pressure. > > Is there some way to have those TMs evenly distributed? Many thanks. > > > >
