Hi Stefano, Essentially the Yarn Session is not much different from a per-job Yarn cluster. In either case, a Flink cluster is brought up with resources provided by Yarn. In case of the Yarn session this cluster doesn't do anything until a job is submitted. In case of the per-job Yarn cluster, a job is immediately submitted after startup and the cluster is shutdown once that job has been completed. That's all.
We're currently working on integrating proper resource allocation into the JobManager. As of now, everything is static, i.e. the JobManager won't allocate more than the initial requested resources. Cheers, Max On Mon, Mar 7, 2016 at 1:38 PM, Stefano Baghino <stefano.bagh...@radicalbit.io> wrote: > Hello everybody, > > I'm currently studying how the Flink/YARN integration works. Right now I'm a > little confused about the practical difference in having a long-running > session on which one deploys several jobs or deploying these jobs > individually. > > My intuition (which may not be correct) is that in the former ase a pool of > resources is allocated to the session and then the usage of these is handled > by the Flink JobManager, while in the latter case the resource allocation is > handled directly by YARN on a per-job basis. Am I right? > > If what I said is (more or less) right and apart from security concerns > (which have been discussed in a previous thread), are there any further > practical differences between having a long-running session or letting YARN > handle jobs? > > Thank you in advance! > > -- > BR, > Stefano Baghino > > Software Engineer @ Radicalbit
