Re: Restoring lost JobManager deployment

2025-01-14 Thread Nic Townsend
Thinking a bit more - is this a bug with fabric8? Assuming the presence of the config map means the JM will try to load HA, and without the map it starts a job from savepoint dir. The config map exists - but the fabric8 configmap check is returning false, so the operator won't create the deploy

Restoring lost JobManager deployment

2025-01-14 Thread Nic Townsend
Hi all, I’m looking for some help understanding a behaviour of the k8s operator when recovering a JobManager deployment. Scenario is – ApplicationCluster, with HA enabled and upgrade mode of savepoint. The job is in failed state – so it’s not in the HA config map. * If I delete the JM pod

Re: [Flink kubernetes operator] custom dependent resources?

2025-01-14 Thread Andrew Otto
+1 to FlinkDeployment helm chart approach. Here is Wikimedia Foundation's chart: (it is heavily dependent on other WMF templates, so you probably can't just use it...but maybe!) https://gerrit.wikimedia.org/r/plugins/gitiles/operations/deployment-charts/+/refs/heads/master/charts/flink-app/ > he

Re: [Flink kubernetes operator] custom dependent resources?

2025-01-14 Thread Nikola Milutinovic
I’d go for a Helm chart, where I would define FlinkDeployment, with desired pod templates and all other stuff. That other stuff can be your headless service, PVCs, CofigMaps,… In the pod template you can define also sidecars and init containers, should you need them. Nix. From: Gyula Fóra Dat

Issues reading kafka avro records with ARRAY field from FlinkSQL table.

2025-01-14 Thread Guillermo Ortiz Fernández
I have created the example table to read data that follows the schema defined in example.avro. > *CREATE TABLE example (* * fieldA STRING,* * fieldB INT,* * arrayField ARRAY* *) WITH (* *'connector' = 'kafka',* *'topic' = 'test_flink',* *'properties.bootstrap.servers' = '***

Re: [Flink kubernetes operator] custom dependent resources?

2025-01-14 Thread Gyula Fóra
Hi Yarden! You have different possibilities: - Use a podTemplate for your FlinkDeployments - Extend and build a custom operator version with your specific requirements - Use a mutating webhook to automatically create a podTemplate for every FlinkDeployment Cheers, Gyula On Sun, Dec 22, 2024 at 2