Hi Fuyao, Currently, Flink only supports perjob mode for Yarn. The standalone job cluster has been replaced with standalone application mode after FLIP-85[1]. Both standalone Flink on K8s and native K8s integration do not support per-job mode.
In your attached video, it is a PoC implementation for presentation. We have introduced the Kubernetes application mode in release 1.11 and agree to not have a per-job cluster support. The only reason is that it is not very convenient to ship the job graphs, user jars, artifacts in Kubernetes environment. [1]. https://cwiki.apache.org/confluence/display/FLINK/FLIP-85+Flink+Application+Mode Best, Yang Fuyao Li <fuyao...@oracle.com> 于2021年4月13日周二 上午8:10写道： > Hello Community, Yang, > > > > I noticed a conflict in the document for per-job mode support for > Kubernetes. > > In the doc here [1], it mentions > > in a Flink Job Cluster, the available cluster manager (like YARN or > Kubernetes) is used to spin up a cluster for each submitted job and this > cluster is available to that job only. > > It implies per job mode is supported in Kubernetes. > > > > However, in the docs [2] and [3], it clearly points out per-job mode is > not supported in Kubernetes. > > > > *This is a conflict statement and is kind of misleading. If needed, I can > create an MR to delete the statement in [1] for Kubernetes.. It is a small > fix. * > > > > I also noticed another thing in the video [4] at 25:08. Yang, you are > executing a command with -e kubernetes-per-job flag. I tried and found such > command is not supported in Flink distribution at all. I noticed the > version you are using is 1.11-snapshot during the demo. Are you modifying > the source code and generated an internal version of Flink….? > > > > > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.12/concepts/flink-architecture.html#flink-job-cluster > > [2] > https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/standalone/kubernetes.html#per-job-cluster-mode > > [3] > https://ci.apache.org/projects/flink/flink-docs-release-1.12/deployment/resource-providers/native_kubernetes.html#per-job-cluster-mode > > [4] https://www.youtube.com/watch?v=pdFPr_VOWTU&t=833s > > > > Best, > > Fuyao >
