Re: observedGeneration field in FlinkDeployment

2023-09-08 Thread Gyula Fóra
Actually, I just realized, the last fully reconciled spec generation should be written into a metadata JSON inside the lastReconciledSpec. So this is already available. For example: lastReconciledSpec: '{"spec":{...},"resource_metadata":{"apiVersion":" flink.apache.org/v1beta1 ","metadata":{"gener

Re: observedGeneration field in FlinkDeployment

2023-09-08 Thread Gyula Fóra
Hi! The lastReconciledSpec field serves similar purpose . We also use the generation in parts of the logic but not generically as observed generation . Could you give an example where this would be useful in addition to what we already have? Thanks Gyula On Sat, 9 Sep 2023 at 02:17, Tony Chen w

Re: Failure to restore from last completed checkpoint

2023-09-08 Thread Alexis Sarda-Espinosa
Hello, Just a shot in the dark here, but could it be related to https://issues.apache.org/jira/browse/FLINK-32241 ? Such failures can cause many exceptions, but I think the ones you've included aren't pointing to the root cause, so I'm not sure if that issue applies to you. Regards, Alexis. On

Re: using CheckpointedFunction on a keyed state

2023-09-08 Thread Krzysztof Chmielewski
My apologies Mattthias, you are right. The issue was that I was trying to access state value from open/init methods where there was not key context. Regarding the CheckpointedFunction interface. From javadoc example and description I got an impression that this can be used to access keyed state on

observedGeneration field in FlinkDeployment

2023-09-08 Thread Tony Chen
Hi Flink Community, I noticed that there is no status.observedGeneration field in the FlinkDeployment spec. The closest field to this is status.reconciliationStatus.lastReconciledSpec. Are there plans to add the observedGeneration field in the spec? This field will be helpful in detecting changes

Re: Failure to restore from last completed checkpoint

2023-09-08 Thread Jacqlyn Bender via user
Hi Yanfei, We were never able to restore from a checkpoint, we ended up restoring from a savepoint as fallback. Would those logs suggest we failed to take a checkpoint before the job manager restarted? Our observabillity monitors showed no failed checkpoints. Here is an exception that occurred be

Re: Problem when testing table API in local

2023-09-08 Thread Oscar Perez via user
quick update, after adding flink-clients it *worked *the first time I ran the test but then the second time got the same error. Looks like a race condition or transient error.? Would love to get some hints on how to troubleshoot this one, if possible. Thanks in advance! Oscar On Fri, 8 Sept 2023

Re: Problem when testing table API in local

2023-09-08 Thread Oscar Perez via user
Hei, Tried adding flink-cients like this: still same error :( implementation "org.apache.flink:flink-clients:${flinkVersion}" On Fri, 8 Sept 2023 at 16:30, Alexey Novakov wrote: > Hi, > > You would need to add the flink-clients module when running in local mode. > The *flink-clients* dependenc

Re: Problem when testing table API in local

2023-09-08 Thread Alexey Novakov via user
Hi, You would need to add the flink-clients module when running in local mode. The *flink-clients* dependency is only necessary to invoke the Flink program locally (for example to run it standalone for testing and debugging). Best regards, Alexey On Fri, Sep 8, 2023 at 3:17 PM Oscar Perez via us

Problem when testing table API in local

2023-09-08 Thread Oscar Perez via user
Hei flink community, We are facing an issue with flink 1.15, 1.16 or 1.16.2 (I tried these 3 versions with same results, maybe it is more general) I am trying to test table API in local and for that I have the following dependencies in my job. See the list of dependencies at the bottom of this em