Re: Issue with flink-kubernetes-operator not updating execution.savepoint.path after savepoint deletion

2023-09-22 Thread Gyula Fóra
Hi Operator savepoint retention and savepoint upgrades have nothing to do with each other I think. Retention is only for periodic savepoints triggered by the operator itself. I would upgrade to the latest 1.6.0 operator version before investigating further. Cheers Gyula On Sat, 23 Sep 2023 at

Re: Issue with flink-kubernetes-operator not updating execution.savepoint.path after savepoint deletion

2023-09-22 Thread Nathan Moderwell
Small update on this. I see that the issue is that we use `upgradeMode: savepoint`, but have not configured the operator to retain savepoints for long enough (the previous operator we used never deleted savepoints so we didn't run into this). I am reconfiguring to use `upgradeMode: last-state` and

Issue with flink-kubernetes-operator not updating execution.savepoint.path after savepoint deletion

2023-09-22 Thread Nathan Moderwell
Hi flink-kubernetes-operator maintainers, We have recently migrated to the official operator and seeing a new issue where our FlinkDeployments can fail and crashloop looking for a non-existent savepoint. On further inspection, the job is attempting to restart from the savepoint specified in execut

Parallel Count Metrics relation

2023-09-22 Thread Kenan Kılıçtepe
Hi, I have a job with parallelism=8. I had some performance issues and I increased the parallelism . It was ok till 10 but when I set it to 12 or something bigger I noticed that, no performance metric such as busy,backpressure or idle metrics are working on Flink UI. If I set parallelism, after w

Side outputs documentation

2023-09-22 Thread Alexis Sarda-Espinosa
Hello, very quick question, the documentation for side outputs states that an OutputTag "needs to be an anonymous inner class, so that we can analyze the type" (this is written in a comment in the example). Is this really true? I've seen many examples where it's a static element and it seems to wo

Flink File Source: File read strategy

2023-09-22 Thread Kirti Dhar Upadhyay K via user
Hi Community, I am using Flink File Source with Amazon S3. Please help me on below questions- 1. When Split Enumerator assigns split to Source Reader, does it downloads the file temporarily and then starts reading/decoding the records from file or it creates direct stream with S3? 1. I