Is Flink HIPAA certified

2020-06-27 Thread Prasanna kumar
Hi Community , Could anyone let me know if Flink is used in US healthcare tech space ? Thanks, Prasanna.

Distributed Anomaly Detection using MIDAS

2020-06-27 Thread Shivin Srivastava
Hi All, I have recently been exploring MIDAS: an algorithm for Streaming Anomaly Detection. A production level parallel and distributed implementation of MIDAS should be quite useful to the industry. I feel that Flink is very well-suited for the same as MIDAS deals with streaming data. If anyone i

Optimal Flink configuration for Standalone cluster.

2020-06-27 Thread Dimitris Vogiatzidakis
Hello, I'm having a bit of trouble understanding the memory configuration on flink. I'm using flink10.0.0 to read some datasets of edges and extract features. I run this on a cluster consisting of 4 nodes , with 32cores and 252GB Ram each, and hopefully I could expand this as long as I can add ext

Re: Question about Watermarks within a KeyedProcessFunction

2020-06-27 Thread David Anderson
With an AscendingTimestampExtractor, watermarks are not created for every event, and as your job starts up, some events will be processed before the first watermark is generated. The impossible value you see is an initial value that's in place until the first real watermark is available. On the ot

Re: Running Apache Flink on the GraalVM as a Native Image

2020-06-27 Thread Stephen Connolly
On Thu 25 Jun 2020 at 12:48, ivo.kn...@t-online.de wrote: > Whats up guys, > > > > I'm trying to run an Apache Flink Application with the GraalVM Native > Image but I get the following error: (check attached file) > > > > I suppose this happens, because Flink uses a lot of low-level-code and is >

Re: Heartbeat of TaskManager timed out.

2020-06-27 Thread Xintong Song
Hi Ori, Here are some suggestions from my side. - Probably the most straightforward way is to try increasing the timeout to see if that helps. You can leverage the configuration option `heartbeat.timeout`[1]. The default is 50s. - It might be helpful to share your configuration setups

Re: Optimal Flink configuration for Standalone cluster.

2020-06-27 Thread Xintong Song
Hi Dimitris, Regarding your questions. a) For standalone clusters, the recommended way is to use `.flink.size` rather than `.process.size`. `.process.size` includes JVM metaspace and overhead in addition to `.flink.size`, which usually do not really matter for standalone clusters. b) In case of di

Re: CEP use case !

2020-06-27 Thread Benchao Li
Hi Aissa, Flink CEP is an api that processes multi-event matching with a pattern, like (START MIDDLE+ END). If you can calculate the "sensor_status" by one record, I think Flink DataStream API / Table & SQL API could satisfy your requirement already. Aissa Elaffani 于2020年6月25日周四 下午11:35写道: > He

Re: Native K8S IAM Role?

2020-06-27 Thread Yang Wang
Hi kevin, If you mean to add annotations for Flink native K8s session pods, you could use "kubernetes.jobmanager.annotations" and "kubernetes.taskmanager.annotations"[1]. However, they are only supported from release-1.11. Maybe you could wait for a little bit more time, 1.11 will be released soon