Re: Queries regarding Flink upgrade strategies

2021-08-31 Thread Amit Bhatia
Hi Matthias, Thanks for the confirmation. @Yang Wang : Any comments from your side ? Regards, Amit On Fri, Aug 27, 2021 at 7:27 PM Matthias Pohl wrote: > Thanks for clarifying that, Amit. Rolling updates with JobManagers and > TaskManagers coming from different Flink versions in the same Fli

Re: Queries regarding Flink upgrade strategies

2021-08-27 Thread Matthias Pohl
Thanks for clarifying that, Amit. Rolling updates with JobManagers and TaskManagers coming from different Flink versions in the same Flink cluster is not supported. @Yang Wang Do you have any recommendations you could share in this regard? Best, Matthias On Fri, Aug 27, 2021 at 2:44 PM Amit Bha

Re: Queries regarding Flink upgrade strategies

2021-08-27 Thread Amit Bhatia
Hi Matthias, What you mention is a little tricky. When we create a new cluster it will have its own volume (PVC) so sending savepoint/checkpoint data from volume (PVC) of the older cluster to the newer cluster is a manual task. Also not sure if savepoint/checkpoint data needs to be copied to the

Re: Queries regarding Flink upgrade strategies

2021-08-27 Thread Matthias Pohl
The upgrade approach mentioned in my previous answer should also work in the context of k8s and pods: Creating a Flink cluster having the newer version should be done before migrating the job using a savepoint. But maybe, I misunderstand your question. Do you have something in mind where you upgrad

Re: Queries regarding Flink upgrade strategies

2021-08-27 Thread Amit Bhatia
Hi Matthias, Thanks for the information but this upgrade is looking like on native (physical/virtual) deployment. I want to understand the upgrade strategies on kubernetes deployments where Flink is running in pods. If you could help in that area it would be great. Regards, Amit Bhatia On Thu, A

Re: Queries regarding Flink upgrade strategies

2021-08-26 Thread Matthias Pohl
Hi Amit, upgrading Flink versions means that you should stop your jobs with a savepoint first. A new cluster with the new Flink version can be deployed next. Then, this cluster can be used to start the jobs from the previously created savepoints. Each job should pick up the work from where it stopp