Thanks zhaobo for the proposal. We've met with a lot of big data users who are trying to migrate their business to flink on kubernetes to improve job performance and cluster utilization. Unfortunately so far kubernetes lacks batch scheduling capabilities compared to traditional batch systems like yarn, slurm and so forth. The customized kubernetes scheduler support will fill these gaps and speed up the migration of flink on kubernetes for users.
Best, William bo zhaobo <bzhaojyathousa...@gmail.com> 于2022年7月7日周四 09:15写道： > Hi, all. > > I would like to raise a discussion in Flink dev ML about Support Customized > Kubernetes Schedulers. > Currentlly, Kubernetes becomes more and more polular for Flink Cluster > deployment, and its ability is better, especially, it supports customized > scheduling. > Essentially, in high-performance workloads, we need to apply new scheduling > policies for meeting the new requirements. And now Flink native Kubernetes > solution is using Kubernetes default scheduler to work with all scenarios, > the default scheduling policy might be difficult to apply in some extreme > cases, so > we need to improve the Flink Kubernetes for coupling those Kubernetes > customized schedulers with Flink native Kubernetes, provides a way for > Flink > administrators or users to use/apply their Flink Clusters on Kubernetes > more flexibility. > > The proposal will introduce the customized K8S schdulers plugin mechanism > and a reference implementation 'Volcano' in Flink. More details see [1]. > > Looking forward to your feedback. > > [1] > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-250%3A+Support+Customized+Kubernetes+Schedulers+Proposal > > Thanks, > BR >
