Hi Dom. I believe this is for per job scenario. That means you create flink cluster in yarn cluster when you submitting your flink. Since this is is per job scenario (there's only one job in this flink cluster), so that it makes more sense to set up the parallelism to be workers * number of slots per worker. Otherwise either you can not get enough slots if the parallelism in flink-conf.yaml is too large or some idle containers will be left there if parallelism in flink-conf.yaml is too small.
Dominik Wosiński <wos...@gmail.com> 于2019年7月20日周六 上午5:36写道: > Hey, > > I was wondering about the relation between the parallelism set by YARN in > Yarn properties file. Currently, as far as I know there is only one > execution of `writeYarnPropertiesFIle` method and it sets the parallelism > in the YARN properties to the number of workers * number of slots per > worker. But doesn't the `flink-conf.yaml` take the precedence in resolving > the configuration ? I am trying to understand the reasoning between always > setting the Yarn properties to the max available slots and whether this > will be used at all, since there is a default value in flink config for > paralellism. > > Thanks in advance, > Best Regards, > Dom. > -- Best Regards Jeff Zhang