Re: How to know when a pipeline ends

2023-05-15 Thread Alexey Novakov via user
Hi Luke, Did you try to use *env.registerJobListener()* to be notified on job execution? https://coderstea.in/post/big-data/flink-job-listener-run-a-task-after-flink-job-is-completed/ Best regards, Alex On Fri, May 12, 2023 at 8:01 PM Luke Xiong wrote: > Hi Weihua and Shammon, > > Thanks for

Re: How to know when a pipeline ends

2023-05-13 Thread Jiadong lu
Hi Luke. I hope this email finds you well. I wanted to share my agreement with Shammon's solution regarding your query. Additionally, I would like to provide some helpful hints that might assist you further: 1. To create a PackagedProgram, you can utilize the PackagedProgram.Builder class. 2

Re: How to know when a pipeline ends

2023-05-12 Thread Luke Xiong
Hi Weihua and Shammon, Thanks for the pointers.I tried both, unfortunately neither works. By enabling "execution.attached", there doesn't seem to be any difference than the default settings. doSomeCleanupTasks() is called right away while the pipeline is still running; and env.executeAsync().getJ

Re: How to know when a pipeline ends

2023-05-11 Thread Weihua Hu
Hi, Luke You can enable "execution.attached", then env.execute() will wait until the job is finished. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#execution-attached Best, Weihua On Fri, May 12, 2023 at 8:59 AM Shammon FY wrote: > Hi Luke, > > Maybe you ca

Re: How to know when a pipeline ends

2023-05-11 Thread Shammon FY
Hi Luke, Maybe you can get 'JobClient' after submit the job and check the job status with 'JobClient.getJobStatus()' Best, Shammon FY On Fri, May 12, 2023 at 2:58 AM Luke Xiong wrote: > Hi, > > My flink job needs to do something when the pipeline execution has ended. > The job code is like th