Adding @Holden Karau <[email protected]> @Dongjoon Hyun <[email protected]> @wuyi <[email protected]> FYI
On Tue, 30 Nov 2021 at 17:46, Yikun Jiang <[email protected]> wrote: > Hey everyone, > > I'd like to start a discussion on "Support Volcano/Alternative Schedulers > Proposal". > > This SPIP is proposed to make spark k8s schedulers provide more YARN like > features (such as queues and minimum resources before scheduling jobs) that > many folks want on Kubernetes. > > The goal of this SPIP is to improve current spark k8s scheduler > implementations, add the ability of batch scheduling and support volcano as > one of implementations. > > Design doc: > https://docs.google.com/document/d/1xgQGRpaHQX6-QH_J9YV2C2Dh6RpXefUpLM7KGkzL6Fg > JIRA: https://issues.apache.org/jira/browse/SPARK-36057 > Part of PRs: > Ability to create resources https://github.com/apache/spark/pull/34599 > Add PodGroupFeatureStep: https://github.com/apache/spark/pull/34456 > > Regards, > Yikun >
