The use of zookeeper in flink

2022-12-28 Thread spoon_lz
Hi All, We use zookeeper to achieve high availability of jobs. Recently, a failure occurred in our flink cluster. It was due to the abnormal downtime of the zookeeper service that all the flink jobs using this zookeeper all occurred failover. The failover startup of a large number of jobs in a s

Re: Sending descriptive alerts from Flink

2022-12-28 Thread Krzysztof Chmielewski
Hi, I manged to accomplish this by using side outputs [1] and stream union [2]. We used those in my past projects where we were using exclusively Java Streaming API to create alerting stream. Every operator was able to produce a side output stream in case of an exception/alert. Later we union all

Sending descriptive alerts from Flink

2022-12-28 Thread Pratham Kumar P via user
Hello, We have a query regarding our use case. Use Case: We need to send descriptive alerts from Flink operators triggered based on certain criteria. Queries: 1. Is there any mechanism provided by Flink, which can be used to send these alerts? 2. We considered using the Async operators,

Re: Failing to build Flink 1.9 using Scala 2.12

2022-12-28 Thread Milind Vaidya
Hi David, Thanks for this useful information. This unblocked and now locally build is successful which was not the case earlier. I have a few more questions though. - Are there any requirements for the maven and / or plugin version ? - Another error says *Failure to find org.apache.kafka:k