Hi Stephan, Thanks for the explanation.
>From your explanation it looks like Flink runs in a mode similar to Spark's YARN-Client mode. Regards, Sourav On Mon, Jan 11, 2016 at 8:27 AM, Stephan Ewen <se...@apache.org> wrote: > Hi! > > Flink is different than Spark in that respect. The driver program in Flink > can submit a program to the master (in YARN Application Master) and > disconnect then. It is not a part of the distributed execution - that is > coordinated only by the master (JobManager). > The driver can stay connected to receive progress updates, though. > > For programs that do consist of multiple parallel executions (that have > count() or collect() statements), the driver needs to stay connected, > because it needs to pull the intermediate results. However, they are all > pulled/proxied through the master (JobManager), so the driver needs not be > able to connect to the workers. The only requirement for firewalled > clusters is to have two ports from the master node reachable by the client. > > Greetings, > Stephan > > > On Mon, Jan 11, 2016 at 5:18 PM, Sourav Mazumder < > sourav.mazumde...@gmail.com> wrote: > >> I am going through the documentation of integrating Flink with YARN. >> >> However not sure whether Flink can be run on YARN in two modes (like >> Spark). In one mode the driver/client program of Flink is also managed by >> YARN. In the second mode where the client program is outside the control of >> YARN. Is the running Flinkon behind Firewalls is like the second mode >> >> Any clarification on this ? >> >> Regards, >> Sourav >> > >
