Flink 1.15 Interval Join error after Deduplicate

2022-10-14 Thread liebingyu
I had a problem with Interval Join after using Deduplicate. I'm using Flink version 1.15. I want to use Flink's Interval Join for double-stream association, and my first table needs to be de-duplicated. Here is my sample code. ``` CREATE TEMPORARY TABLE `source` ( id INT, name STRING, eve

Re: question about Async IO

2022-10-14 Thread Krzysztof Chmielewski
Hi Galen, i will tell from my experience as a Flink user and developer of Flink jobs. *"if the input to an AsyncFunction is a keyed stream, can I assume that all input elements with the same key will be handled by the same instance of the async operator"* >From what I know (and someone can corre

question about Async IO

2022-10-14 Thread Galen Warren
I have a question about Flink's Async IO support: Async I/O | Apache Flink . I understand that access to state is not supported in an AsyncFunction. However, if the input to an AsyncFunction is a keyed st

SQL Changes between 1.14 and 1.15?

2022-10-14 Thread PACE, JAMES
We've noticed the following difference in sql when upgrading from flink 1.14.5 to 1.15.2 around characters that are escaped in an sql statement: This statement: tableEnvironment.executeSql("select * from testTable WHERE lower(field1) LIKE 'b\"cd\"e%'"); produces a runtime error in flink 1.15.2

Re: Sometimes checkpoints to s3 fail

2022-10-14 Thread Matthias Pohl via user
Hi Evgeniy, is it Ceph which you're using as a S3 server? All the Google search entries point to Ceph when looking for the error message. Could it be that there's a problem with the version of the underlying system? The stacktrace you provided looks like Flink struggles to close the File and, there

Re: jobmaster's fatal error will kill the session cluster

2022-10-14 Thread Jie Han
Thanks for the note. The root cause is the following Caused by: org.apache.flink.util.FlinkRuntimeException: Failed to start the operator coordinators at org.apache.flink.runtime.scheduler.DefaultOperatorCoordinatorHandler.startOperatorCoordinators(DefaultOperatorCoordinatorHandler.java:

Re: jobmaster's fatal error will kill the session cluster

2022-10-14 Thread Matthias Pohl via user
Hi Jie Han, welcome to the community. Just a little side note: These kinds of questions are more suitable to be asked in the user mailing list. The dev mailing list is rather used for discussing feature development or project-related topics. See [1] for further details. About your question: The st

Broadcast state restoration for BroadcastProcessFunction

2022-10-14 Thread Alexis Sarda-Espinosa
Hello, I wrote a test for a broadcast function to check how it handles broadcast state during retries [1] (the gist only shows a subset of the test in Kotlin, but it's hopefully understandable). The test will not pass unless my function also implements CheckpointedFunction, although those interfac