Thanks a lot for driving this discussion Yang Wang. I believe Flink's K8s integration will become more and more important as more users switch to K8s. Hence, it is very important that Flink gives users a very K8s experience and can cover many different use cases.
Cheers, Till On Sun, Jan 19, 2020 at 11:46 AM felixzheng zheng <felixzhen...@gmail.com> wrote: > Hi Yang Wang, > > Thanks for your effort on this topic and inviting me, I am glad to join the > future work on native Kubernetes integration, will try to join the slack > channel latter. > > Yang Wang <danrtsey...@gmail.com> 于2020年1月19日周日 下午6:04写道： > > > Hi Canbin Zheng, > > > > I have found that you created some tickets about Flink on Kubernetes. We > > just have the same requirements. > > Maybe we could have more discussion about the use cases and > implementation > > details. I have created > > a slack channel[1], please join in if you want. > > > > Any dev users if you have some ideas about the new features. Join in us, > > let's make Flink on Kubernetes > > moving forward together. > > > > > > [1]. > > > > > https://slack.com/share/ISUKQ2WG5/cXwlw5HZgp3AE5ElE8m580Pk/enQtOTEyNjcwMDk4NTQ5LWZiNTU0ZmJiZTU4MWU1OTk2YjllNmE0OTg2YjIxYjA2YmE1MWFlOTE4NWFhMDBkMzE4NDQzYjk1YmQwMDI2MzU > > > > > > Yang Wang <danrtsey...@gmail.com> 于2020年1月19日周日 下午5:07写道： > > > > > Hi everyone, > > > > > > > > > Currently Flink supports the resource management system YARN and Mesos. > > > However, they were not > > > designed for fast moving cloud native architectures, and they could not > > > support mixed workloads (e.g. batch, > > > streaming, deep learning, web services, etc.) relatively well. At the > > same > > > time, Kubernetes is evolving very > > > fast to fill those gaps and become the de-facto orchestration > framework. > > > So running Flink on Kubernetes is > > > a very basic requirement for many users. > > > > > > > > > At least, we have the following advantages when natively running Flink > on > > > Kubernetes. > > > * Flink KubernetesResourceManager will allocate TaskManager pods > > > dynamically based on the resource > > > requirement of the jobs. > > > * Using Flink bundled scripts to start/stop session cluster on > > > Kuberenetes. Do not need external tools anymore. > > > * Compared with Yarn deployment, different Flink clusters could get > > better > > > isolation by leveraging the ability > > > of Kubernetes. > > > > > > > > > Recently, i also find more and more uses are very interested in running > > > Flink on Kubernetes natively. > > > The community has already made some efforts[1] and will be released in > > > 1.10. Welcome to have a taste and > > > give us your feedback. However, it is a basic requirement and we still > > > need many features before production. > > > So i want to start this discussion to collect the requirements that you > > > have came across. Feel free to share > > > your valuable thoughts. We will try to conclude and create sub tasks in > > > the umbrella ticket[2]. Also i will move > > > some existing tickets there for easier tracking them. > > > > > > > > > Best, > > > Yang > > > > > > [1]. > > > > > > https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/native_kubernetes.html > > > [2]. https://issues.apache.org/jira/browse/FLINK-14460 > > > > > >
