Re: Securing metrics reporters (Flink <-> Prometheus)

Hello, this is currently not supported. Regards, Chesnay On 02.03.2018 18:38, Fritz Budiyanto wrote: Hi All, How can I configure encryption for metrics.reports ? Particularly Prometheus ? I do not see any mention of encryption in the metrics.reports traffic in flink documentation. Is encry

Re: Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

Hi sundy, 1. Some partition of your input kafka don't have data. Since window watermark is the min value of all it's inputs, if there are no data from one of it's inputs, window will never be triggered. You can set parallelism of your job to 1 to avoid this problem(PS: Maybe this bug is fixed now,

Re: Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

Hi, thanks for your reply. I have searched it in stackoverflow, and there is someone who has the some problem. https://stackoverflow.com/questions/40993753/flink-streaming-program-runs-correctly-with-processing-time-but-will-not-produc

Re: Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

Hi, for periodically generated watermarks, you should use `ExecutionConfig.setAutoWatermarkInterval()` to set an interval. Hope that helps. Best, Xingcan > On 3 Mar 2018, at 12:08 PM, sundy <543950...@qq.com> wrote: > > > > Hi, I got a problem in Flink and need your help. > > I tried to u

Watermark not worked as expected in TimeCharacteristic.EvenTime setting.

Hi, I got a problem in Flink and need your help. I tried to use TimeCharacteristic.EvenTime, but the sink function never be executed. public class StreamingJob { public static void main(String[] args) throws Exception { // set up the streaming execution environment final StreamExe

Re: Which test cluster to use for checkpointing tests?

Hi Stephan, Thanks for the update. So is support for “running checkpoints with closed sources” part of FLIP-15 , or something separate? Regards, — Ken > On Mar 1, 2018, at 9:07 AM, Stephan Ewen wrote: > > @Ken The i

Serialization and Deserialization of Avro messages stored in Kafka

Hello, I have a few topics that I want to read from Kafka, which consist mainly on a key value pair of: timestamp (key) and value (byte array). The bite array doesn't really have a class to deserialize from, since the Avro Record we have comes from a "SELECT * FROM..." that selects several SQL ta

Securing metrics reporters (Flink <-> Prometheus)

Hi All, How can I configure encryption for metrics.reports ? Particularly Prometheus ? I do not see any mention of encryption in the metrics.reports traffic in flink documentation. Is encryption supported? If yes, could shed a light not how to do this ? metrics.reporters: prom metrics.reporte

Restart hook and checkpoint

All, It looks like Flink's default behavior is to restart all operators on a single operator error - in my case it is a Kafka Producer timing out. When this happens, I see logs that all operators are restarted. This essentially leads to data loss. In my case the volume of data is so high that it

Re: Fwd: Flink throwing ava.lang.ClassNotFoundException after upgrading to 1.4.1

Hi Ankit, This is a known issue in 1.4.1. Please see  https://issues.apache.org/jira/browse/FLINK-8741. The release for 1.4.2 will include a fix for this issue, and we already have a release candidate being voted at the moment. Hopefully, it will be released soon, probable early next week. Chee

Fwd: Flink throwing ava.lang.ClassNotFoundException after upgrading to 1.4.1

Hey Guys, I just want to throw a question regarding the latest flink release 1.4.1. I have a flink topology which was first written using flink version 1.2.0. Since then, we are continuously try to keep our libraries upto date. So, we try to upgrade this same flink topology from version 1.4.0 to