Re: HybridSource permanently failed after restoring from checkpoint

2022-08-04 Thread Benenson, Michael via user
I have created Jira https://issues.apache.org/jira/browse/FLINK-28817 Please, assign it to me, I have a fix that I’m testing in my environment From: Benenson, Michael via user Date: Tuesday, August 2, 2022 at 5:47 PM To: mas.chen6...@gmail.com , user@flink.apache.org Cc: Deshpande, Omkar , Ros

Re: Flink task lifecycle listener/hook/SPI

2022-08-04 Thread Jonathan Weaver
I think the piece you are missing is you cannot guarantee where the function will run in general. It may get sent to several different task executors, and each executor may not be on the same machine or JVM so the code has to init once distributed at least once. You have to think that every functi

Re: Flink task lifecycle listener/hook/SPI

2022-08-04 Thread Allen Zoo
Thanks a lot! In our scenario, doing init in open function or at static block is not good as excepted. 1. it is too late, we expect the init will happen in a task init stage, means init it even before the open was called method. 2. it is not reusable or not convenient for end user, we have manny

Flink fails at stop operation after migration from 1.14 to 1.15.1

2022-08-04 Thread Vararu, Vadim
Hi all, Trying to migrate a job from Flink 1.14 to 1.15.1. The job itself consumes from a kinesis stream and writes to s3. The stop operation works well on 1.14, however, on 1.15.1 it fails (both with and without savepoint). The jobs fails with different exceptions when there is data flowing th