[ANNOUNCE] Apache Flink 1.14.6 released

2022-09-27 Thread Xingbo Huang
The Apache Flink community is very happy to announce the release of Apache Flink 1.14.6, which is the fifth bugfix release for the Apache Flink 1.14 series. Apache Flink® is an open-source stream processing framework for distributed, high-performing, always-available, and accurate data streaming a

RE: Loading broadcast state on BroadcastProcessFunction instantiation or open method

2022-09-27 Thread Schwalbe Matthias
Hi Alfredo, Did you consider implementing org.apache.flink.streaming.api.checkpoint.CheckpointedFunction interface in your broadcast function … the initializeState(…) function should give you access to the state backend. Kind regards Thias From: David Anderson Sent: Tuesday, September 27,

Re: Sorting by source event time

2022-09-27 Thread Noel OConnor
yep that worked, thanks for the help On Tue, Sep 27, 2022 at 3:05 AM yuxia wrote: > > You can change to "order by eventTIme". And it should work. > > You can sort on event time, but it must be time-ascending-order without > 'limit'. > If you still want to a descending order, I think you can try

Re: Re: INSERT INTO will work faster in Flink than in regular database?

2022-09-27 Thread podunk
I was thinking if I can use Flink to process large files and save result to another file or database (jdbc). So loading file into Flink table and then loop through rows inserting result to some temp Flink table. Then save result to jdbc table or file.   Sent: Thursday, September 22, 2022 at 5

Serialization in window contents and network buffers

2022-09-27 Thread Alexis Sarda-Espinosa
Hi everyone, I know the low level details of this are likely internal, but at a high level we can say that operators usually have some state associated with them. Particularly for error handling and job restarts, I imagine windows must persist state, and operators in general probably persist netwo

Re: Loading broadcast state on BroadcastProcessFunction instantiation or open method

2022-09-27 Thread David Anderson
Logically it would make sense to be able to initialize BroadcastState in the open method of a BroadcastProcessFunction, but in practice I don't believe it can be done -- because the necessary Context isn't made available. Perhaps you could use the State Processor API to bootstrap some state into t