Hi Thomas, Thanks a lot for the great efforts in this well-organized FLIP! After reading the FLIP carefully, I think Yang has given some great feedback and I just want to share some of my concerns: # 1 Flink Native vs Standalone integration I believe it is reasonable to support both modes in the long run but in the FLIP and previous thread[1], it seems that we have not made a decision on which one to implement initially. The FLIP mentioned "Maybe start with support for Flink Native" for reusing codes in [2]. Is it the selected one finally? # 2 K8S StatefulSet v.s. K8S Deployment In the CR Example, I notice that the kind we use is FlinkDeployment. I would like to check if we have made the decision to use K8S Deployment workload resource. As the name implies, StatefulSet is for stateful apps while Deployment is usually for stateless apps. I think it is worthwhile to consider the choice more carefully due to some user case in gcp operator[3], which may influence our other design choices(like the Flink application deletion strategy).
Again, thanks for the work and I believe this FLIP is pretty useful for many customers and I hope I can make some contributions to this FLIP impl! Best regard, Biao Geng [1] https://lists.apache.org/thread/l1dkp8v4bhlcyb4tdts99g7w4wdglfy4 [2] https://github.com/wangyang0918/flink-native-k8s-operator [3] https://github.com/GoogleCloudPlatform/flink-on-k8s-operator/pull/354 Yang Wang <danrtsey...@gmail.com> 于2022年1月26日周三 15:25写道： > Thanks Thomas for creating FLIP-212 to introduce the Flink Kubernetes > Operator. > > The proposal looks already very good to me and has integrated all the input > in the previous discussion(e.g. native K8s VS standalone, Go VS java). > > I read the FLIP carefully and have some questions that need to be > clarified. > > # How do we run a Flink job from a CR? > 1. Start a session cluster and then followed by submitting the Flink job > via rest API > 2. Start a Flink application cluster which bundles one or more Flink jobs > It is not clear enough to me which way we will choose. It seems that the > existing google/lyft K8s operator is using #1. But I lean to #2 in the new > introduced K8s operator. > If #2 is the case, how could we get the job status when it finished or > failed? Maybe FLINK-24113[1] and FLINK-25715[2] could help. Or we may need > to enable the Flink history server[3]. > > > # ApplicationDeployer Interface or "flink run-application" / > "kubernetes-session.sh" > How do we start the Flink application or session cluster? > It will be great if we have the public and stable interfaces for deployment > in Flink. But currently we only have an internal interface > *ApplicationDeployer* to deploy the application cluster and > no interfaces for deploying session cluster. > Of cause, we could also use the CLI command for submission. However, it > will have poor performance when launching multiple applications. > > > # Pod Template > Is the pod template in CR same with what Flink has already supported[4]? > Then I am afraid not the arbitrary field(e.g. cpu/memory resources) could > take effect. > > > [1]. https://issues.apache.org/jira/browse/FLINK-24113 > [2]. https://issues.apache.org/jira/browse/FLINK-25715 > [3]. > > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/advanced/historyserver/ > [4]. > > https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/resource-providers/native_kubernetes/#pod-template > > > > Best, > Yang > > > Thomas Weise <t...@apache.org> 于2022年1月25日周二 13:08写道： > > > Hi, > > > > As promised in [1] we would like to start the discussion on the > > addition of a Kubernetes operator to the Flink project as FLIP-212: > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-212%3A+Introduce+Flink+Kubernetes+Operator > > > > Please note that the FLIP is currently focussed on the overall > > direction; the intention is to fill in more details once we converge > > on the high level plan. > > > > Thanks and looking forward to a lively discussion! > > > > Thomas > > > > [1] https://lists.apache.org/thread/l1dkp8v4bhlcyb4tdts99g7w4wdglfy4 > > >
