At the moment, this requirement has not come up very often. In general, I am always a bit cautious when adding functionality which executes user code in the JobManager because it can easily become a stability problem. On the other hand, I can't think of a different solution other than polling the job status atm.
Cheers, Till On Fri, Jan 8, 2021 at 1:23 PM Aljoscha Krettek <aljos...@apache.org> wrote: > Till or Chesnay (cc'ed), have you thought about adding a hook on the > JobMaster/JobManager to allow external systems to get push notifications > about submitted jobs. > > If they are ok with such as future, would you maybe be interested in > implementing it yourself, Wenhao? > > Best, > Aljoscha > > On 2020/09/28 11:14, 季文昊 wrote: > >Hi Aljoscha, > > > >Yes, that is not enough, since the `JobListener`s are called only once > when > >`excute()` or `executeAsync()` is called. And in order to sync the status, > >I also have to call `JobClient#getJobStatus` periodically. > > > >On Fri, Sep 25, 2020 at 8:12 PM Aljoscha Krettek <aljos...@apache.org> > >wrote: > > > >> Hi, > >> > >> I understand from your email that > >> `StreamExecutionEnvironment.registerJobListener()` would not be enought > >> for you because you want to be notified of changes on the cluster side, > >> correct? That is when the job status changes on the master. > >> > >> Best, > >> Aljoscha > >> > >> On 23.09.20 14:31, 季文昊 wrote: > >> > Hi there, > >> > > >> > I'm working on a Flink platform in my corp, which provides a service > to > >> > provision and manage multiple dedicated Flink clusters. The problem is > >> that > >> > we want to sync a job status without delay after its submission > through > >> our > >> > platform as long as it has been changed. > >> > > >> > Since we want to update this in-time and make our services stateless, > >> > pulling a job's status periodically is not a good solution. I do not > find > >> > any proper way to achieve this by letting a job manager push changes > >> > directly to our platform except changing the source code, which > registers > >> > an additional `JobStatusListener` in the method > >> > `org.apache.flink.runtime.jobmaster.JobMaster#startScheduling`. > >> > > >> > I wonder if we can enhance `JobStatusListener` a little bit so that a > >> Flink > >> > user can register his custom JobStatusListener at the startup. > >> > > >> > To be specific, we can have a `JobStatusListenerFactory` interface and > >> its > >> > corresponding `ServiceLoader<JobStatusListenerFactory>`, where > >> > the JobStatusListenerFactory will have the following method: > >> > - JobStatusListener createJobStatusListener(Properties properties); > >> > > >> > Custom listeners will be created during the JobMaster#startScheduling > >> > method. > >> > > >> > If someone would like to implement his own JobStatusListener, he will > >> > package all the related classes into a standalone jar with a > >> > > >> > `META-INF/services/org.apache.flink.runtime.executiongraph.JobStatusListener` > >> > file and place it under the `lib/` directory. > >> > > >> > In addition, I find that there is a Jira ticket similar to what I'm > >> > asking: FLINK-17104 but I do not see any comment or update yet. Hope > >> anyone > >> > could help me move on this feature or give me some suggestions about > it. > >> > > >> > Thanks, > >> > Wenhao > >> > > >> > >> >
