Typically, when one wants to run a Flink job on a Hadoop YARN installation,
one creates a Yarn session (e.g. ./bin/yarn-session.sh -n 4 -qu
test-yarn-queue) and runs intended Flink job(s) (e.g. ./bin/flink run -c
MyFlinkApp -m job-manager-host:job-manager-port <overriding app config
params> myapp.jar) on the Flink cluster whose job manager URL is returned
by the previous command.

My questions are:
- Does yarn-session.sh need conf/flink-conf.yaml to be available in Flink
installation on every container in YARN? If this file is needed, how can
one run different YARN sessions (with potentially very different
configurations) on the same Hadoop YARN installation simultaneously?
- Is it possible to start the YARN session programmatically? If yes, I
believe I should look at classes like YarnClusterClient
<https://ci.apache.org/projects/flink/flink-docs-stable/api/java/org/apache/flink/yarn/YarnClusterClient.html>.
Is that right? Is there any other guidance on how to do this
programmatically (e.g. I have a management UI that wants to start/stop YARN
sessions and deploy Flink jobs to it)?

Regards,
Kedar

Reply via email to