Great to have other integrations and improved K8s support. Left some comments/questions in the design doc. Tom On Tuesday, November 30, 2021, 02:46:42 AM CST, Yikun Jiang <yikunk...@gmail.com> wrote: Hey everyone,
I'd like to start a discussion on "Support Volcano/Alternative Schedulers Proposal". This SPIP is proposed to make spark k8s schedulers provide more YARN like features (such as queues and minimum resources before scheduling jobs) that many folks want on Kubernetes. The goal of this SPIP is to improve current spark k8s scheduler implementations, add the ability of batch scheduling and support volcano as one of implementations. Design doc: https://docs.google.com/document/d/1xgQGRpaHQX6-QH_J9YV2C2Dh6RpXefUpLM7KGkzL6FgJIRA: https://issues.apache.org/jira/browse/SPARK-36057Part of PRs: Ability to create resources https://github.com/apache/spark/pull/34599 Add PodGroupFeatureStep: https://github.com/apache/spark/pull/34456 Regards,Yikun
