Hi Anuj, By "standalone job on yarn", I assume you mean running one job per Flink cluster on Yarn, which is also known as job mode, or per-job mode? I'm asking because Flink has another standalone deployment mode [1], aside from the Yarn deployment mode.
1. The major difference between Flink Application Cluster (a.k.a. job mode) [2] and Flink Session Cluster [3] is whether you can execute only multiple jobs per Flink cluster. A session cluster allows you to execute multiple jobs in the same Flink cluster, sharing/reusing the process resources. You might want to run a session cluster for various purposes. One example is to reduce the resource overhead of Flink Master processes. When you have multiple jobs with very small scale, having one Master process per job could become a significant overhead. Another use case is when you need to rapidly submit new jobs (e.g., interactive analysis), you may want to leverage the pre-exist task manager processes to speed up the initial process. 2. AFAIK, the REST API should work for both application & session clusters, except for that you cannot submit new jobs to an application cluster since it's designed for executing only one job. Thank you~ Xintong Song [1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/deployment/cluster_setup.html [2] https://ci.apache.org/projects/flink/flink-docs-release-1.10/concepts/glossary.html#flink-application-cluster [3] https://ci.apache.org/projects/flink/flink-docs-release-1.10/concepts/glossary.html#flink-session-cluster On Mon, Jun 8, 2020 at 9:39 PM aj <ajainje...@gmail.com> wrote: > I am running some stream jobs that are long-running always. I am currently > submitting each job as a standalone job on yarn. > > 1. I need to understand what is the advantage of using yarn-session and > when should I use that. > 2. Also, I am not able to access rest API services is it because I am > running as standalone job over yarn. Is REST API works only in yarn-session? > > > -- > Thanks & Regards, > Anuj Jain > > > <http://www.cse.iitm.ac.in/%7Eanujjain/> >
