Hey Gil, I'm referring to when a pod exits on its own, not when being deleted. Deployments only support the "Always" restart policy [1].
In my understanding, the JM only cleans up HA data when it is shutdown[2], after which the process will exit which leads to the problem with k8s Deployment restart policies. Best, Austin [1]: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#pod-template [2]: https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/ops/rest_api/#cluster On Wed, Sep 7, 2022 at 4:43 PM Gil De Grove <gil.degr...@euranova.eu> wrote: > Hello Austin, > > I'm not aware of any limitations of deployement not letting pod exit > (correctly or incorrectly). What do you mean by that exactly? Would it be > possible for you to point out to piece of documentation that make you think > that ? > > A pod, if correctly setup will be exited when receiving it's sigterm or > sigkill from the orchestrator. > So when "deleting" the deployment, the pods are quitted correctly. In the > case flink did triggered a savepoint before, you can then restart from that > savepoint. > Usually, when a pod is not being terminated this means that the SIG is not > transferred to the correct process. > > Hopes this helps. > > Regards, > Gil > > > On Wed, Sep 7, 2022, 21:16 Austin CawleyEdwards <austin.caw...@gmail.com> > wrote: > >> Cool, thanks! How does it clean up the HA data, if the cluster is never >> able to shut down (due to the k8s Deployment restriction)? >> >> Best, >> Austin >> >> On Mon, Sep 5, 2022 at 6:51 PM Gyula Fóra <gyula.f...@gmail.com> wrote: >> >>> Hi! >>> >>> The operator supports both Flink native and standalone deployment modes >>> and in both cases the JM is deployed as k8s Deployment. >>> >>> During upgrade Flink/operator deletes the deployment after savepoint and >>> waits for termination before it creates a new one with the updated spec. >>> >>> Cheers, >>> Gyula >>> >>> On Mon, 5 Sep 2022 at 07:41, Austin Cawley-Edwards < >>> austin.caw...@gmail.com> wrote: >>> >>>> Hey Marco, >>>> >>>> Unfortunately there is no built in k8s API that models an application >>>> mode JM exactly but Deployments should be fine, in general. As Gyula notes, >>>> where they can be difficult is during application upgrades as Deployments >>>> never let their pods exit, even if successful, so there is no way to stop >>>> the cluster gracefully. >>>> >>>> Is stopping your application with a savepoint and redeploying a >>>> workable solution for image upgrades? In this way a Job could still be >>>> used. >>>> >>>> >>>> @Gyula, how are JMs handled in the operator? Job, Deployment, or >>>> something custom? >>>> >>>> >>>> Best, >>>> Austin >>>> >>>> >>>> >>>> On Mon, Sep 5, 2022 at 6:15 AM Gyula Fóra <gyula.f...@gmail.com> wrote: >>>> >>>>> You can use deployments of course , the operator and native k8s >>>>> integration does exactly that. >>>>> >>>>> Even then job updates can be tricky so I believe you are much better >>>>> off with the operator. >>>>> >>>>> Gyula >>>>> >>>>> On Sun, 4 Sep 2022 at 11:11, marco andreas <marcoandreas...@gmail.com> >>>>> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> Thanks for the response, I will take a look at it. >>>>>> >>>>>> But if we aren't able to use the flink operator due to technical >>>>>> constraints is it possible to deploy the JM as deployment without any >>>>>> consequences that I am not aware of? >>>>>> >>>>>> Sincerely, >>>>>> >>>>>> Le sam. 3 sept. 2022 à 23:27, Gyula Fóra <gyula.f...@gmail.com> a >>>>>> écrit : >>>>>> >>>>>>> Hi! >>>>>>> You should check out the Flink Kubernetes Operator. I think that >>>>>>> covers all your needs . >>>>>>> >>>>>>> >>>>>>> https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-main/ >>>>>>> >>>>>>> Cheers, >>>>>>> Gyula >>>>>>> >>>>>>> On Sat, 3 Sep 2022 at 13:45, marco andreas < >>>>>>> marcoandreas...@gmail.com> wrote: >>>>>>> >>>>>>>> >>>>>>>> We are deploying a flink application cluster on k8S. Following the >>>>>>>> official documentation the JM is deployed As a job resource , however >>>>>>>> we >>>>>>>> are deploying a long running flink job that is not supposed to be >>>>>>>> terminated and also we need to update the image of the flink job. >>>>>>>> >>>>>>>> The problem is that the job is an immutable resource, we >>>>>>>> cant update it. >>>>>>>> >>>>>>>> So I'm wondering if it's possible to use a deployment resource for >>>>>>>> the jobmanager and if there will be any side effects or repercussions. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>> Gil De Grove >
