Re: Flink task lifecycle listener/hook/SPI

2022-08-04 Thread Jonathan Weaver
I think the piece you are missing is you cannot guarantee where the function will run in general. It may get sent to several different task executors, and each executor may not be on the same machine or JVM so the code has to init once distributed at least once. You have to think that every functi

Re: Flink task lifecycle listener/hook/SPI

2022-08-04 Thread Allen Zoo
Thanks a lot! In our scenario, doing init in open function or at static block is not good as excepted. 1. it is too late, we expect the init will happen in a task init stage, means init it even before the open was called method. 2. it is not reusable or not convenient for end user, we have manny

Re: Flink task lifecycle listener/hook/SPI

2022-08-03 Thread Lijie Wang
Hi Allen, >From my experience, you can do your init setup by the following 2 ways: 1. Do your init setup in RichFunction#open method, see [1] for details. 2. Do your init setup in static block, it will be executed when the class is loaded. [1] https://nightlies.apache.org/flink/flink-docs-master/

Flink task lifecycle listener/hook/SPI

2022-08-02 Thread Allen Zoo
Hi all, We went to do some init env setup before the flink task run, And we have noticed the Task Lifecycle | Apache Flink doc described, but we can't find listener/hook/SPI interface do some custom init jobs bef