Hi all, Thanks for the FLIP. I have a couple of remarks/questions:
* Regarding the motivation, it mentions that the development trend is that Flink supports both batch and stream processing. I think the vision and trend is that we have unified batch- and stream processing. What I'm missing is the vision on what's the impact for customized Kubernetes schedulers on stream processing. Could there be some elaboration on that? * While the FLIP talks about customized schedulers, it focuses on Volcano. Why is the choice made to only focus on Volcano and not on other schedulers like Apache YuniKorn? Can we not also provide an implementation for YuniKorn at the same time? Spark did the same with SPARK-36057 [1] * We still have quite a lot of tech debt on testing for Kubernetes [2]. I think that this FLIP would be a great improvement for Flink, but I am worried that we will add more tech debt to those tests. Can we somehow improve this situation? Best regards, Martijn [1] https://issues.apache.org/jira/browse/SPARK-36057 [2] https://issues.apache.org/jira/browse/FLINK-20392 Op wo 13 jul. 2022 om 04:11 schreef 王正 <cswangzh...@gmail.com>: > +1 > > On 2022/07/07 01:15:13 bo zhaobo wrote: > > Hi, all. > > > > I would like to raise a discussion in Flink dev ML about Support > Customized > > Kubernetes Schedulers. > > Currentlly, Kubernetes becomes more and more polular for Flink Cluster > > deployment, and its ability is better, especially, it supports > customized > > scheduling. > > Essentially, in high-performance workloads, we need to apply new > scheduling > > policies for meeting the new requirements. And now Flink native > Kubernetes > > solution is using Kubernetes default scheduler to work with all > scenarios, > > the default scheduling policy might be difficult to apply in some extreme > > cases, so > > we need to improve the Flink Kubernetes for coupling those Kubernetes > > customized schedulers with Flink native Kubernetes, provides a way for > Flink > > administrators or users to use/apply their Flink Clusters on Kubernetes > > more flexibility. > > > > The proposal will introduce the customized K8S schdulers plugin mechanism > > and a reference implementation 'Volcano' in Flink. More details see [1]. > > > > Looking forward to your feedback. > > > > [1] > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-250%3A+Support+Customized+Kubernetes+Schedulers+Proposal > > > > Thanks, > > BR > >
