Re: [DISCUSS] Flink checkpointInfo and FlinkStateSnapshots

2024-12-19 Thread Mate Czagany
epointInfo > savepointHistory[] . With 1.10 version as per documentation, savepointInfo > fields in Flink resource CR are deprecated as by default > kubernetes.operator.snapshot.resource.enabled is enabled, then how do we > get savepointHistory information. > > Regards > Laj

Re: [DISCUSS] Flink checkpointInfo and FlinkStateSnapshots

2024-12-18 Thread Mate Czagany
Hi Michal, I am happy that you have found this new feature interesting, I hope you will find it useful if you plan to use it. 1. I am not sure when or how the deprecated fields will be removed, it should happen when the community is satisfied with the new FlinkStateSnapshots CRDs. For checkpoints

Re: [VOTE] Release flink-connector-hbase v4.0.0, release candidate #1

2024-11-13 Thread Mate Czagany
+1 (non-binding) - Built from source with JDK 11 - Verified checksum and signatures - Verified source code does not contain binaries - Verified artifacts deployed to the maven repository - Verified no binaries in source release - Reviewed flink-web PR Thank you for driving this release Ferenc! B

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.10.0, release candidate #1

2024-10-21 Thread Mate Czagany
+1 (non-binding) - Verified checksums and signatures - Verified no binary files - Verified all source files contain Apache license header - Suspended job with 1.9.0 Operator, upgraded to 1.10.0, restarted job - Verified manual/periodic/upgrade FlinkStateSnapshot resources However with `helm upgra

[jira] [Created] (FLINK-36577) Add compatibility check for FlinkStateSnapshot CRD

2024-10-21 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-36577: Summary: Add compatibility check for FlinkStateSnapshot CRD Key: FLINK-36577 URL: https://issues.apache.org/jira/browse/FLINK-36577 Project: Flink Issue

Re: Resource limits for the k8s

2024-10-11 Thread Mate Czagany
Hi, You can use the following configuration options to set resource limits on Kubernetes, please see more info on the configuration docs [1] - kubernetes.jobmanager.cpu.limit-factor - kubernetes.jobmanager.memory.limit-factor - kubernetes.taskmanager.cpu.limit-factor - kubernetes.taskmanager.memor

[jira] [Created] (FLINK-36467) Bump google-java-format and spotless

2024-10-10 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-36467: Summary: Bump google-java-format and spotless Key: FLINK-36467 URL: https://issues.apache.org/jira/browse/FLINK-36467 Project: Flink Issue Type: Improvement

Re: [DISCUSS] FLIP-XXX: Introduce CLI API compatibility guarantees

2024-10-09 Thread Mate Czagany
Hi Ferenc, Thank you for creating this FLIP, having some kind of CLI deprecation process would be very useful for developers and users. The FLIP does a good job explaining the deprecation of actions, but there aren't any mentions of command options. I think it would add some value to also include

[jira] [Created] (FLINK-36037) Add compability for 1.9 last savepoint info on upgrade/recovery

2024-08-12 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-36037: Summary: Add compability for 1.9 last savepoint info on upgrade/recovery Key: FLINK-36037 URL: https://issues.apache.org/jira/browse/FLINK-36037 Project: Flink

[jira] [Created] (FLINK-35640) Drop Flink 1.15 support for the operator

2024-06-18 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35640: Summary: Drop Flink 1.15 support for the operator Key: FLINK-35640 URL: https://issues.apache.org/jira/browse/FLINK-35640 Project: Flink Issue Type

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.9.0, release candidate #1

2024-06-18 Thread Mate Czagany
Hi, +1 (non-binding) Note: Using the Apache Flink KEYS file [1] to verify the signatures your key seems to be expired, so that file should be updated as well. - Verified checksums and signatures - Built source distribution - Verified all pom.xml versions are the same - Verified install from RC r

Re: [VOTE] FLIP-464: Merge "flink run" and "flink run-application"

2024-06-12 Thread Mate Czagany
Hi, Thank you for driving this Ferenc, +1 (non-binding) Regards, Mate Ferenc Csaky ezt írta (időpont: 2024. jún. 12., Sze, 17:23): > Hello devs, > > I would like to start a vote about FLIP-464 [1]. The FLIP is about to > merge back the > "flink run-application" functionality to "flink run", so

Re: Savepoints not considered during failover

2024-06-07 Thread Mate Czagany
Hi, I think this was last discussed in FLIP-193 [1] where the reasoning is mostly the same as Matthias said, savepoints are owned by the user and Flink cannot depend on them. In older versions Flink also took savepoints into account when restoring a job, with the possibility to skip savepoints by

[jira] [Created] (FLINK-35493) Make max history age and count configurable for FlinkStateSnapshot resources

2024-05-30 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35493: Summary: Make max history age and count configurable for FlinkStateSnapshot resources Key: FLINK-35493 URL: https://issues.apache.org/jira/browse/FLINK-35493 Project

[jira] [Created] (FLINK-35492) Add metrics for FlinkStateSnapshot resources

2024-05-30 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35492: Summary: Add metrics for FlinkStateSnapshot resources Key: FLINK-35492 URL: https://issues.apache.org/jira/browse/FLINK-35492 Project: Flink Issue Type: Sub

Re: [DISCUSS] FLIP-XXX Add K8S conditions to Flink CRD

2024-05-30 Thread Mate Czagany
Hi, I would definitely keep this as a FLIP. Not all FLIPs have to be big changes, and this format makes it easier for others to chime in and follow. I am not a Kubernetes expert, but my understanding is that we don't have to follow any strict convention for the type names in the conditions, e.g.

Re: [DISCUSS] Merge "flink run" and "flink run-application" in Flink 2.0

2024-05-29 Thread Mate Czagany
Hi Ferenc, Thanks for the FLIP, +1 from me for the proposal. I think these changes would be a great solution to all the confusion that comes from these two action parameters. Best regards, Mate Ferenc Csaky ezt írta (időpont: 2024. máj. 28., K, 16:13): > Thank you Xintong for your input. > > I

[RESULT][VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-30 Thread Mate Czagany
Hi all, I'm happy to announce that FLIP-446: Kubernetes Operator State Snapshot CRD [1] has been accepted with 9 approving votes (4 binding) [2]. I would like to thank everyone who took part in the discussion and/or voted. - Rui Fan (binding) - Ferenc Csaky (non-binding) - Gyula Fora (binding) -

[jira] [Created] (FLINK-35267) Create documentation for FlinkStateSnapshot CR

2024-04-29 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35267: Summary: Create documentation for FlinkStateSnapshot CR Key: FLINK-35267 URL: https://issues.apache.org/jira/browse/FLINK-35267 Project: Flink Issue Type

[jira] [Created] (FLINK-35266) Add e2e tests for FlinkStateSnapshot CRs

2024-04-29 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35266: Summary: Add e2e tests for FlinkStateSnapshot CRs Key: FLINK-35266 URL: https://issues.apache.org/jira/browse/FLINK-35266 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-35265) Implement FlinkStateSnapshot custom resource

2024-04-29 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35265: Summary: Implement FlinkStateSnapshot custom resource Key: FLINK-35265 URL: https://issues.apache.org/jira/browse/FLINK-35265 Project: Flink Issue Type: Sub

[jira] [Created] (FLINK-35263) FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-29 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35263: Summary: FLIP-446: Kubernetes Operator State Snapshot CRD Key: FLINK-35263 URL: https://issues.apache.org/jira/browse/FLINK-35263 Project: Flink Issue Type

[VOTE] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-24 Thread Mate Czagany
Hi everyone, I'd like to start a vote on the FLIP-446: Kubernetes Operator State Snapshot CRD [1]. The discussion thread is here [2]. The vote will be open for at least 72 hours unless there is an objection or insufficient votes. [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP-446%3A+

Re: [DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-20 Thread Mate Czagany
der if I am missing any good argument against the single > FlinkStateSnapshot here. > > Cheers, > Gyula > > > On Fri, Apr 19, 2024 at 9:09 PM Mate Czagany wrote: > >> Hi Robert and Thomas, >> >> Thank you for sharing your thoughts, I will try to address your

Re: [DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-19 Thread Mate Czagany
avepointPath > > > > > > One minor comment: > > > > "/** Dispose the savepoints upon CRD deletion. */" > > > > I think this should be "upon CR deletion", not "CRD deletion". > > > > Thanks again for this great FLIP!

Re: [DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-16 Thread Mate Czagany
alled or not because we don't want to break operator startup. > > Gyula > > On Tue, Apr 16, 2024 at 4:48 PM Mate Czagany wrote: > > > Hi Ferenc, > > > > Thank you for your comments, I have updated the Google docs with a new > > section for the new confi

Re: [DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-16 Thread Mate Czagany
y kind of confusion. > > Best, > Ferenc > > > > On Tuesday, April 16th, 2024 at 11:34, Mate Czagany > wrote: > > > > > > > Hi Everyone, > > > > I would like to start a discussion on FLIP-446: Kubernetes Operator State > > Snapshot CRD. > &g

[DISCUSS] FLIP-446: Kubernetes Operator State Snapshot CRD

2024-04-16 Thread Mate Czagany
Hi Everyone, I would like to start a discussion on FLIP-446: Kubernetes Operator State Snapshot CRD. This FLIP adds a new custom resource for Operator users to create and manage their savepoints and checkpoints. I have also developed an initial POC to prove that this approach is feasible, you can

[jira] [Created] (FLINK-35106) Kubernetes Operator ignores checkpoint type configuration

2024-04-15 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-35106: Summary: Kubernetes Operator ignores checkpoint type configuration Key: FLINK-35106 URL: https://issues.apache.org/jira/browse/FLINK-35106 Project: Flink

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.8.0, release candidate #1

2024-03-20 Thread Mate Czagany
Hi, +1 (non-binding) - Verified checksums - Verified signatures - Verified no binaries in source distribution - Verified Apache License and NOTICE files - Executed tests - Built container image - Verified chart version and appVersion matches - Verified Helm chart can be installed with default val

Re: [VOTE] FLIP-439: Externalize Kudu Connector from Bahir

2024-03-20 Thread Mate Czagany
+1 (non-binding) Thank you, Mate Ferenc Csaky ezt írta (időpont: 2024. márc. 20., Sze, 15:11): > Hello devs, > > I would like to start a vote about FLIP-439 [1]. The FLIP is about to > externalize the Kudu > connector from the recently retired Apache Bahir project [2] to keep it > maintainable

Re: [DISCUSS] Manual savepoint triggering in flink-kubernetes-operator

2024-03-12 Thread Mate Czagany
Hi, I really like this idea as well, I think it would be a great improvement compared to how manual savepoints currently work, and suits Kubernetes workflows a lot better. If there are no objections, I can investigate it during the next few weeks and see how this could be implemented in the curre

[jira] [Created] (FLINK-34439) Move chown operations to COPY commands in Dockerfile

2024-02-13 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-34439: Summary: Move chown operations to COPY commands in Dockerfile Key: FLINK-34439 URL: https://issues.apache.org/jira/browse/FLINK-34439 Project: Flink Issue

[jira] [Created] (FLINK-34438) Kubernetes Operator doesn't wait for TaskManager deletion in native mode

2024-02-13 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-34438: Summary: Kubernetes Operator doesn't wait for TaskManager deletion in native mode Key: FLINK-34438 URL: https://issues.apache.org/jira/browse/FLINK-34438 Pr

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.7.0, release candidate #1

2023-11-16 Thread Mate Czagany
+1 (non-binding) - Checked signatures, checksums - No binaries found in the source release - Verified all source files contain the license header - All pom files point to the correct version - Verified Helm chart version and appVersion - Verified Docker image tag - Ran flink-autoscaler-standalone

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.1, release candidate #1

2023-10-23 Thread Mate Czagany
+1 (non-binding) - Verified checksums, signatures, no binary found in source - Verified Helm chart and Docker images - Tested autoscaler on 1.18 with reactive scaling Regards, Mate Gyula Fóra ezt írta (időpont: 2023. okt. 23., H, 9:45): > +1 (binding) > > - Verified checksums, signatures, sour

Re: [VOTE] Apache Flink Kubernetes Operator Release 1.6.0, release candidate #1

2023-07-30 Thread Mate Czagany
+1 (non-binding) - Verified checksums and signatures - Found no binaries in source - Helm chart points to correct docker image - Installed via remote helm repo - Reactive example up- and down-scaled with- and without reactive mode - Autoscale with Kafka source - HA stateful deployment, with savepo

[jira] [Created] (FLINK-32652) Operator cannot scale standalone deployments in reactive mode

2023-07-23 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-32652: Summary: Operator cannot scale standalone deployments in reactive mode Key: FLINK-32652 URL: https://issues.apache.org/jira/browse/FLINK-32652 Project: Flink

Re: what happened to the images in FLIP-24: SQL Client?

2023-04-04 Thread Mate Czagany
st regards, > Jing > > On Tue, Apr 4, 2023 at 8:17 PM Mate Czagany wrote: > > > Hi, > > > > It does not work for me either, the images are hosted on Google Drawings > > and it seems like it went private. > > Maybe you have the images in your browser cache

Re: what happened to the images in FLIP-24: SQL Client?

2023-04-04 Thread Mate Czagany
Hi, It does not work for me either, the images are hosted on Google Drawings and it seems like it went private. Maybe you have the images in your browser cache if you can see them. I see that other FLIPs have their images uploaded to Confluence, I think that would be a better way to store them. R

[jira] [Created] (FLINK-31613) Some default operator config values are overwritten by values.yaml

2023-03-25 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-31613: Summary: Some default operator config values are overwritten by values.yaml Key: FLINK-31613 URL: https://issues.apache.org/jira/browse/FLINK-31613 Project: Flink

[jira] [Created] (FLINK-31326) Disabled source scaling breaks downstream scaling if source busyTimeMsPerSecond is 0

2023-03-05 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-31326: Summary: Disabled source scaling breaks downstream scaling if source busyTimeMsPerSecond is 0 Key: FLINK-31326 URL: https://issues.apache.org/jira/browse/FLINK-31326

[jira] [Created] (FLINK-31187) Standalone HA mode does not work if dynamic properties are supplied

2023-02-22 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-31187: Summary: Standalone HA mode does not work if dynamic properties are supplied Key: FLINK-31187 URL: https://issues.apache.org/jira/browse/FLINK-31187 Project: Flink

[jira] [Created] (FLINK-30899) FileSystemTableSource with CSV format incorrectly selects fields if filtering for partition

2023-02-04 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-30899: Summary: FileSystemTableSource with CSV format incorrectly selects fields if filtering for partition Key: FLINK-30899 URL: https://issues.apache.org/jira/browse/FLINK-30899

[jira] [Created] (FLINK-30812) YARN with S3 resource storage fails for Hadoop 3.3.2

2023-01-28 Thread Mate Czagany (Jira)
Mate Czagany created FLINK-30812: Summary: YARN with S3 resource storage fails for Hadoop 3.3.2 Key: FLINK-30812 URL: https://issues.apache.org/jira/browse/FLINK-30812 Project: Flink Issue