Additional question, is there any plan in the Flink community to provide an easier way of deploying Flink with application mode on YARN? Overall I feel application mode would be most widely used when it comes to running multiple long-running flink jobs.
Thanks Leon On Sun, May 22, 2022 at 3:49 PM Leon Xu <l...@attentivemobile.com> wrote: > Thanks Geng, I will take a look at the YARNApplicationITCase > implementation. > And yeah I am switching from session mode to application mode, as > application mode can offer better resource isolation. > Besides that with application mode we don't need to worry about the issue > of Metaspace not getting GCed. > > Leon > > On Sat, May 21, 2022 at 8:11 PM Geng Biao <biaoge...@gmail.com> wrote: > >> Hi Leon, >> >> >> >> If you are using application mode through Flink on YARN, the codes in >> YARNApplicationITCase may be helpful, which has similar function calls with >> CliFrontend but may be more straightforward. >> >> >> >> Side notes: in my own experience of using Flink, it is not a very good >> idea to run a flink jar using java codes through application mode. >> >> If you are using session modes, there could be some RESTful interfaces >> which makes life much easier. But when using application mode, the cluster >> is launched after running commands like `flink run-application -t >> yarn-application ….` so REST services are not initially available. As a >> result, you have to fight with some deployment details like setting >> environment variables, making classloaders work correctly and so on. All >> details shaded by flink have to be handled by hand. >> >> >> >> Best, >> >> Biao Geng >> >> >> >> *发件人**:* Leon Xu <l...@attentivemobile.com> >> *日期**:* 星期六, 2022年5月21日 下午2:24 >> *收件人**:* user <user@flink.apache.org> >> *主题**:* Application mode deployment through API call >> >> Hi Flink community, >> >> >> >> I am looking to deploy my flink job through *Application Mode *in my >> Java program. Ideally I'd like my java code to just call an API to achieve >> this. Does flink provide a rest API to support this? I don't seem to find >> any documentation or code on that. >> >> If I need to build this on my own, is there any example I can follow? >> >> Should I just follow how CliFrontend.java does ? (I believe the command >> line calls this internally)? Or is there any other better examples? >> >> >> >> >> >> Thanks >> >> Leon >> >
