That mainly depends on how much parallelism of your job. The main bottleneck of job manager usually because it is busy to handle rpc requests and gc. At most time you can set larger jm memory to address it by pass `-jm 4096` to `yarn-session.sh start`.
Best, tison. weilongxing <weilongx...@aicaigroup.com> 于2018年9月20日周四 下午2:29写道： > Thanks. > > I am wondering whether the job manager will be the bottleneck and how many > jobs could a job manager support in session mode. I tried to find the > bottleneck in test environment but failed. > > > 在 2018年9月20日，下午2:16，vino yang <yanghua1...@gmail.com> 写道： > > Hi weilong, > > As you said, there are advantages and disadvantages to each of the two > approaches. > However, I hope you know that the "single job" mode has a huge advantage > over the "YARN flink session" mode in that it provides job-level isolation > (whether JM or TM). > This will allow the Job to be more fine-grained, and the refactoring of > Flink's FLIP-6-based deployment model tends to be "single job" mode. > But it will start more JM (appmaster) and take up more resources. > But in the end, how to choose also requires you to evaluate and weigh. > > Thanks, vino. > > weilongxing <weilongx...@aicaigroup.com> 于2018年9月20日周四 上午10:27写道： > >> There are two methods to deploy flink applications on yarn. The first one >> is use yarn-session and all flink applications are deployed in the session. >> The second method is each flink application deploy on yarn as a yarn >> application. >> >> My question is what's the difference between these two methods? Which one >> to choose in product environment? >> >> I can't find any material about this. >> >> I think the first method will save resources since only need one >> jobmanager(yarn application master). While it is also the disadvantage >> since the only jobmanager can be the bottleneck while flink applications >> getting more and more. >> > >
