+1 for this proposal. Volcano has show its strength in some specified cases, such as ML. And its schedule ability will be enhancement for kubernetes default scheduler.
Look forward to see the integration between Volcano and Flink. Thanks. On 2022/07/08 00:25:45 bo zhaobo wrote: > Thanks Yang Wang. > > <Just to remind that the flink-kubernetes-volcano-*.jar will be an optional > jar located in the $FLINK_HOME/opt. Users who want to try the volcano > scheduler need > to copy it to the plugins directory.r.> > > --- >> > A: Yeah, the said separated jar pkg won't be loaded by default. But when > users copy the jar pkg from $FLINK_HOME/opt/flink-kubernetes-*.jar to > plugins directory, > then the customized scheduler will be load and the functionality will > be enabled. That would be helpful that different users want different > functions if more and more > schedulers will be introduced in the future. > > Thanks, > > BR > > Bo Zhao > > > > Yang Wang <danrtsey...@gmail.com> 于2022年7月7日周四 12:08写道： > > > Thanks zhaobo for starting the discussion and preparing the FLIP. > > > > The customized Kubernetes Schedulers support will be very helpful for the > > users who still hesitates to migrate the Flink workloads from YARN to > > Kubernetes. > > Now leveraging the ability of customized K8s scheduler, many advanced > > scheduling features(e.g. priority scheduling, dynamic resource sharing, > > etc.) could be > > introduced to make the streaming/batch jobs run more smoothly in a shared > > K8s cluster. > > > > Just to remind that the flink-kubernetes-volcano-*.jar will be an optional > > jar located in the $FLINK_HOME/opt. Users who want to try the volcano > > scheduler need > > to copy it to the plugins directory. > > > > > > Best, > > Yang > > > > bo zhaobo <bzhaojyathousa...@gmail.com> 于2022年7月7日周四 09:16写道： > > > > > Hi, all. > > > > > > I would like to raise a discussion in Flink dev ML about Support > > Customized > > > Kubernetes Schedulers. > > > Currentlly, Kubernetes becomes more and more polular for Flink Cluster > > > deployment, and its ability is better, especially, it supports > > customized > > > scheduling. > > > Essentially, in high-performance workloads, we need to apply new > > scheduling > > > policies for meeting the new requirements. And now Flink native > > Kubernetes > > > solution is using Kubernetes default scheduler to work with all > > scenarios, > > > the default scheduling policy might be difficult to apply in some extreme > > > cases, so > > > we need to improve the Flink Kubernetes for coupling those Kubernetes > > > customized schedulers with Flink native Kubernetes, provides a way for > > > Flink > > > administrators or users to use/apply their Flink Clusters on Kubernetes > > > more flexibility. > > > > > > The proposal will introduce the customized K8S schdulers plugin mechanism > > > and a reference implementation 'Volcano' in Flink. More details see [1]. > > > > > > Looking forward to your feedback. > > > > > > [1] > > > > > > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-250%3A+Support+Customized+Kubernetes+Schedulers+Proposal > > > > > > Thanks, > > > BR > > > > > >
