villebro commented on issue #31408: URL: https://github.com/apache/superset/issues/31408#issuecomment-2594269927
> I feel in operator for ease of use each superset architecture component to be managed by its isolated controller in controller runtime, with a single controller (superset-controller) managing all these components with their respective controllers via ownerreference. This would allow isolation and scale the controllers for each component in superset architecture as individual components evolve with superset versions. @Synarcs are you proposing having separate CRDs and controllers for each component, like the web worker, async worker, scheduler etc? In theory this does sound appealing. However, I see the following problems: - The workers currently map fairly cleanly into vanilla `Deployment` resources. Having separate CRDs and controllers for each worker would only add an extra layer to the chain of resources (WebWorker -> Deployment -> ReplicaSet -> Pod) without adding that much value. It would essentially be a worker-specific `Deployment` wrapper. - The workers and scheduler will need identical image tags, environment variables, mounts, service references etc. Having separate CRDs (and consequently CRs) for all these will cause significant duplication and make the management of the cluster as a whole more complex. - Separating the individual components would also make it more logical to split out `Ingress`/`Gateway` and `Service`s. However, the exact structure of these, especially the main service, tend to be uninteresting to the typical admin. Breaking these out would make it more difficult to get a Superset deployment up and running. If any of the above changes in the future, I see no reason why we can't break up the operator into separate CRDs and controllers. However, given the current architecture, I feel having a single CRD is the most convenient means of abstracting Superset as a whole. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
