Re: Custom Trigger for a TimeWindow

2025-02-26 Thread Pedro Mázala
Hey there Antonio! I'm just bringing what we discussed over slack to here. Got it. I think the missing bit may be this. > I'll edit your trigger code and show what I think would work. Non tested > code @Override public TriggerResult onElement(BigquerySnowplow element, long timestamp, W window,

Re: How can we read checkpoint data for debugging state

2025-02-26 Thread Gabor Somogyi
The plan is to support everything but the question is when. Related short term options I would suggest to turn off changelog in the reader and try to read the state as-is. G On Wed, Feb 26, 2025 at 1:22 PM Sachin Mittal wrote: > Hi, > Looks like this class is very minimally implemented. What e

Re: How can we read checkpoint data for debugging state

2025-02-26 Thread Sachin Mittal
Hi, Looks like this class is very minimally implemented. What exactly does the State Processor API support ? Looking at that class it seems to only support prioritizedOperatorSubtaskState. Are there any plans to support changelogs in future. In view of this, what are my options? I wanted to debu

Writing Tests in pyFlink

2025-02-26 Thread mokaspar
Hi Folks, I’d like to secure my pyFlink job with tests. I haven’t found any information writing pytest-tests in the web, so I have read through the test-cases of pyflink itself and concluded, that I could test my KeyedProcessFunctions similar as in https://github.com/apache/flink/blob/master/f

Re: How can we read checkpoint data for debugging state

2025-02-26 Thread Gabor Somogyi
State processor API is now not supporting changelog. Please see here [1]. [1] https://github.com/apache/flink/blob/69559fb5d231d704633fed807773cd1853601862/flink-libraries/flink-state-processing-api/src/main/java/org/apache/flink/state/api/runtime/SavepointTaskStateManager.java#L127 G On Wed, F

Re: How can we read checkpoint data for debugging state

2025-02-26 Thread Sachin Mittal
Hello Folks, I am trying to read the flink checkpointed state and I am getting this error: Caused by: java.io.IOException: Failed to restore state backend at org.apache.flink.state.api.input.StreamOperatorContextBuilder.build( StreamOperatorContextBuilder.java:140) at org.apache.flink.state.api.in

Fwd: Custom Trigger for a TimeWindow

2025-02-26 Thread Antonio Davide Cali via user
Hello team, I'm trying to figure out how to correctly move on this. Scenario, I am using a EventTimeSessionWindows with a fixed gap. I need, also, to trigger a *FIRE_AND_PURGE* for closing a window after a fixed amount of time since the first event received for the window, *no matter what*. So fo

Re: Flink 1.20.0 missing linux/amd64 images

2025-02-26 Thread Alexis Sarda-Espinosa
Hello, I would like to point out that the same has happened with the images for 1.20.1 Regards, Alexis. Am So., 4. Aug. 2024 um 09:45 Uhr schrieb Bjarke Tornager < bjarketorna...@gmail.com>: > Hi Weijie, > > Thanks for looking into this. It looks like the docker-hub repo was > updated with the l