> It's applicable regardless of if the operators are maintained as part of > Spark core or not, with the maturity of Kubernetes features around CRD > support and webhooks. The GCP Spark operator supports a lot of additional > pod/container configs using a webhook, and this approach seems pretty > successful so far. >
Agreed (in fact the existence of >= two independent operator projects testifies to this). I do believe this has implications for how feature requests for spark-on-k8s get fielded here upstream. There's a non-zero amount of cognitive load involved with recommending that a feature request be deferred to some independent operator project. Going forward, will that complicate the upstream story for spark-submit, history server, and shuffle service on a kube backend?
