Hi Canbing,
Thanks a lot for sharing your thoughts to improve the Flink on K8s native integration. Frankly speaking, your discussion title confuses me and i am wondering whether you want to refactor the whole design. However, after i dive into the details and the provided document, i realize that mostly you want to improve the implementation. Regarding your two main points. >> Introduce a unified monadic-step based orchestrator architecture that has a better, cleaner and consistent abstraction for the Kubernetes resources construction process, both applicable to the client side and the master side. When i introduce the decorator for the K8s in Flink, there is always a guideline in my mind that it should be easy for extension and adding new features. Just as you say, we have lots of functions to support and the K8s is also evolving very fast. The current `ConfigMapDecorator`, `FlinkMasterDeploymentDecorator`, `TaskManagerPodDecorator` is a basic implementation with some prerequisite parameters. Of course we could chain more decorators to construct the K8s resources. For example, InitializerDecorator -> OwnerReferenceDecorator -> FlinkMasterDeploymentDecorator -> InitContainerDecorator -> SidecarDecorator -> etc. I am little sceptical about splitting every parameter into a single decorator. Since it does not take too much benefits. But i agree with moving some common parameters into separate decorators(e.g. volume mount). Also introducing the `~Builder` class and spinning off the chaining decorator calls from `Fabric8FlinkKubeClient` make sense to me. >> Add some dedicated tools for centrally parsing, verifying, and managing the Kubernetes parameters. Currently, we always get the parameters directly from Flink configuration( e.g. `flinkConfig.getString(KubernetesConfigOptions.CONTAINER_IMAGE)`). I think it could be improved by introducing some dedicated conf parser classes. It is a good idea. Best, Yang felixzheng zheng <felixzhen...@gmail.com> 于2020年2月21日周五 上午9:31写道： > Hi everyone, > > I would like to kick off a discussion on refactoring the existing > Kubernetes resources construction architecture design. > > I created a design document [1] that clarifies our motivation to do this > and some improvement proposals for the new design. > > Briefly, we would like to > 1. Introduce a unified monadic-step based orchestrator architecture that > has a better, cleaner and consistent abstraction for the Kubernetes > resources construction process, both applicable to the client side and the > master side. > 2. Add some dedicated tools for centrally parsing, verifying, and managing > the Kubernetes parameters. > > It would be great to start the efforts before adding big features for the > native Kubernetes submodule, and Tison and I plan to work together for this > issue. > > Please let me know your thoughts. > > Regards, > Canbin Zheng > > [1] > > https://docs.google.com/document/d/1dFBjqho8IRyNWxKVhFnTf0qGCsgp72aKON4wUdHY5Pg/edit?usp=sharing > [2] https://issues.apache.org/jira/browse/FLINK-16194 >
