Re: Which mode to choose flink on Yarn.

2018-09-19 Thread 陈梓立
That mainly depends on how much parallelism of your job. The main bottleneck of job manager usually because it is busy to handle rpc requests and gc. At most time you can set larger jm memory to address it by pass `-jm 4096` to `yarn-session.sh start`. Best, tison. weilongxing 于2018年9月20日周四 下午

Re: Which mode to choose flink on Yarn.

2018-09-19 Thread weilongxing
> In addition, per job mode calculate resource that the job required, while > session mode require you config a static config of that persistent session. I tested using flink 1.5.2 and found that session mode can also support dynamic resource. You don’t have to config static config. I am wond

Re: Which mode to choose flink on Yarn.

2018-09-19 Thread weilongxing
Thanks. I am wondering whether the job manager will be the bottleneck and how many jobs could a job manager support in session mode. I tried to find the bottleneck in test environment but failed. > 在 2018年9月20日,下午2:16,vino yang 写道: > > Hi weilong, > > As you said, there are advantages and

Re: Which mode to choose flink on Yarn.

2018-09-19 Thread 陈梓立
Hi weilong, As vino said, the main advantage of per job mode is that it provides job-level isolation, and that of session mode is that it set up a persistent session and accept job, which means the overhead of resource request/setup would loose. In addition, per job mode calculate resource that th

Re: Which mode to choose flink on Yarn.

2018-09-19 Thread vino yang
Hi weilong, As you said, there are advantages and disadvantages to each of the two approaches. However, I hope you know that the "single job" mode has a huge advantage over the "YARN flink session" mode in that it provides job-level isolation (whether JM or TM). This will allow the Job to be more