Hi Till,
Since it has been a little bit while, I would like to restart this discussion.
Would you please share some ideas about this? Will it become a
stability problem if we create a "surefire" listener to prevent the
exception thrown from those custom listeners? [1]
By the way, as you have ment
Hi Till,
Thanks for taking time out of your busy schedule.
I have created a POC for this feature.
The code change for the Flink source code will be like this commit:
https://github.com/predatorray/flink/commit/2cab8bb1119162213632db984d2eb7529b8140e7
Generally, the idea is that custom `JobStatusL
Hi Wenhao,
Aljoscha might not be as responsive as before. Surely you can create a POC
to evaluate different approaches. But the outcome should be a design which
we discuss before starting to implement the code properly. At the moment
the community might be a bit busy with the upcoming feature free
Hi Till. Indeed, there is no proper solution now other than the polling
method. It is painful to have such code in our platform since it consumes a
lot of resources to keep the polling run periodically when there are
hundreds of Flink clusters to maintain. A lot of pollings are actually
useless as
Hi Till，
IIUC for application mode, we already allow to run user code in job manager
Till Rohrmann 于2021年1月8日周五 下午9:53写道：
> At the moment, this requirement has not come up very often. In general, I
> am always a bit cautious when adding functionality which executes user code
> in the JobManager
At the moment, this requirement has not come up very often. In general, I
am always a bit cautious when adding functionality which executes user code
in the JobManager because it can easily become a stability problem. On the
other hand, I can't think of a different solution other than polling the
j
Till or Chesnay (cc'ed), have you thought about adding a hook on the
JobMaster/JobManager to allow external systems to get push notifications
about submitted jobs.
If they are ok with such as future, would you maybe be interested in
implementing it yourself, Wenhao?
Best,
Aljoscha
On 2020/0
Hi Aljoscha,
Yes, that is not enough, since the `JobListener`s are called only once when
`excute()` or `executeAsync()` is called. And in order to sync the status,
I also have to call `JobClient#getJobStatus` periodically.
On Fri, Sep 25, 2020 at 8:12 PM Aljoscha Krettek
wrote:
> Hi,
>
> I unde
Hi,
I understand from your email that
`StreamExecutionEnvironment.registerJobListener()` would not be enought
for you because you want to be notified of changes on the cluster side,
correct? That is when the job status changes on the master.
Best,
Aljoscha
On 23.09.20 14:31, 季文昊 wrote:
Hi
Hi there,
I'm working on a Flink platform in my corp, which provides a service to
provision and manage multiple dedicated Flink clusters. The problem is that
we want to sync a job status without delay after its submission through our
platform as long as it has been changed.
Since we want to updat
10 matches
Mail list logo
