Re: High availability data clean up

2021-10-31 Thread Weiqing Yang
Thanks for the info, Yang! I'm using Finalizer and the labels to handle the deletion. On Mon, Oct 25, 2021 at 3:56 AM Yang Wang wrote: > Hi Weiqing, > > > Why does Flink not set the owner reference of HA related ConfigMaps to > JobManager deployment? It is easier to clean up for users. > The maj

Re: High availability data clean up

2021-10-25 Thread Yang Wang
Hi Weiqing, > Why does Flink not set the owner reference of HA related ConfigMaps to JobManager deployment? It is easier to clean up for users. The major reason is that simply deleting the HA related ConfigMaps will make the HA data located in DFS leak. > How to delete the HA ConfigMap from exter

Re: High availability data clean up

2021-10-22 Thread Weiqing Yang
Thanks for the replies, Yangze and Vijay! We are using standalone Flink on K8s (we created a K8s operator to manage the life cycle of the flink clusters (session mode)). Seems there is no way for the operator to know when these HA related configMaps are created (if the operator somehow can know wh

Re: High availability data clean up

2021-10-21 Thread Vijay Bhaskar
In HA mode the configMap will be retained after deletion of the deployment: https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/ha/kubernetes_ha/ ( Refer High availability data clean up) On Fri, Oct 22, 2021 at 8:13 AM Yangze Guo wrote: > For application mode, when the jo

Re: High availability data clean up

2021-10-21 Thread Yangze Guo
For application mode, when the job finished normally or be canceled, the ConfigMaps will be cleanup. For session mode, when you stop the session through [1], the ConfigMaps will be cleanup. [1] https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/resource-providers/native_