Custom Restart Strategy for controlled Exceptions in Flink 1.20

2025-04-22 Thread Aian Cantabrana
Hi, I'm working with Flink 1.20 (DataStream API, java 11) and I have a question regarding the current capabilities for implementing a custom restart strategy . I've gone through the documentation, but it’s not entirely clear whether it's possible in this version to define

Re: Custom restart strategy

2022-05-29 Thread Shengkai Fang
Hi. Maybe the metric reporter[1] is suitabe for your case. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/ unknown unknown 于2022年5月28日周六 12:49写道: > Thanks Shengkai! Unfortunately, this would require querying status for > each job continuously. Given v

Re: Custom restart strategy

2022-05-27 Thread unknown unknown
Thanks Shengkai! Unfortunately, this would require querying status for each job continuously. Given very few pipelines experience failures and they are far in-between, I am looking for a push based model vs polling. Thanks AK On Thu, May 26, 2022 at 7:21 PM Shengkai Fang wrote: > Hi. > > I thin

Re: Custom restart strategy

2022-05-26 Thread Shengkai Fang
Hi. I think you can use REST OPEN API to fetch the job status from the JM periodically to detect whether something happens. Currently REST OPEN API also supports to fetch the exception list for the specified job[2]. Best, Shengkai [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops

Custom restart strategy

2022-05-26 Thread unknown unknown
Hello Users! I would like to notify an external endpoint when a streaming job has a certain number of restarts. While I can use a service to continuously *poll* Flink metrics and identify failing jobs, I am looking to inverse the action and have the job notify. We have around ~50 streaming jo