Re: Resume running a statefull in a different k8s cluster

2025-06-12 Thread Pedro Mázala
ch job failover from k8s cluster to k8s cluster > > our cluster are multi node, multi az but they run critical business > process hence we want to protect against region failure > > On Thu, Jun 12, 2025, at 4:30 PM, Pedro Mázala wrote: > > Using Flink k8s operator, you may use th

Re: Resume running a statefull in a different k8s cluster

2025-06-12 Thread Pedro Mázala
-node cluster may help you. k8s will try to spread the deployments among the different nodes. Even if one dies, it will make sure everything is there due to k8s desired state mechanism. Att, Pedro Mázala Be awesome On Thu, 12 Jun 2025 at 15:52, gustavo panizzo wrote: > Hello > > I run

Re: Regarding increasing Parallelism of the keyby and window function while migrating

2025-05-28 Thread Pedro Mázala
an I increase the parallelism of this keyBy and window combination here. I think using disable chaining before the keyBy then increasing the parallelism of the output of the keyed stream would work out. Att, Pedro Mázala Be awesome On Wed, 28 May 2025 at 10:42, Sambhav Gupta wrote: > Hi a

Re: Regarding increasing Parallelism of the keyby and window function while migrating

2025-05-26 Thread Pedro Mázala
What is the error on the parallelism you're facing? Att, Pedro Mázala Be awesome On Mon, 26 May 2025 at 10:13, Sambhav Gupta wrote: > Hi Team, > > We are migrating our codebase of flink to V2.1 version. Here were using > dataset jobs which we need to migrate to data str

Re: BackendBuildingException when trying to restore heap backend from S3 : The specified key does not exist. (Service: Amazon S3; Status Code: 404...)

2025-05-26 Thread Pedro Mázala
Hello there Jean! This may be related to a network issue between Flink and minio/s3. I had this in the past and I configured Flink to not start if the state was not possible. So every time I received one of those, Flink would restart and try again. Att, Pedro Mázala Be awesome On Thu, 22 May

Re: Working with Flink

2025-05-26 Thread Pedro Mázala
Hello there Bryan! It looks like Flink cannot find the s3 schema in your packages. How are you adding the jars? Is the error happening on TM or on JM? Att, Pedro Mázala Be awesome On Thu, 22 May 2025 at 19:45, Bryan Cantos wrote: > Hello, > > I have deployed the Flink Operator

Re: Custom Trigger for a TimeWindow

2025-02-26 Thread Pedro Mázala
But this may > bring some other pains. Bringing it here so other people may benefit from it :D Thread <https://apache-flink.slack.com/archives/C03G7LJTS2G/p1740498241124159> Att, Pedro Mázala +31 (06) 3819 3814 Be awesome On Wed, 26 Feb 2025 at 10:45, Antonio Davide Cali via user

Re: Resolving Table using Java API

2025-01-30 Thread Pedro Mázala
space and catalog table) and make sure those configs are coming properly and not null/empty. Att, Pedro Mázala +31 (06) 3819 3814 Be awesome On Wed, 29 Jan 2025 at 17:11, Kamesh Sampath wrote: > I am on Apache Flink 1.20, trying to use pure Java SQL API to create a > catalog and table

Re: Performance problem with FlinkSQL

2025-01-30 Thread Pedro Mázala
e smaller topics, which don’t do much but still consume minimal resources even if idle. On the table API I'm not sure if you can choose parallelism per "task" DataStream I'm sure you can do it. Att, Pedro Mázala +31 (06) 3819 3814 Be awesome On Wed, 29 Jan 2025 at 22: