Re: Poor use case? (late arrival + IoT + windowing)

2021-05-15 Thread invalidrecipient
First I was told that my application need only perform keyed aggregation of streaming IoT data on a sliding window. Flink seemed the obvious choice. Then I was told that the window size must be configurable, taking on one of 5 possible values, anywhere from 5-60 minutes. Oh and configuration cha

Re: Flink compatibility with MINIO S3 Operator/Tenant

2021-05-15 Thread Arvid Heise
Hi Robert, we have an end-to-end-test with minio. You have to use s3.path.style.access; I think the actual key depends on the used s3 connector, so just try out both. For reference, the actual test can be found here [2]. [1] https://github.com/apache/flink/blob/master/flink-end-to-end-tests/test-

Re: MemoryStateBackend Issue

2021-05-15 Thread Milind Vaidya
Sounds good. How do I achieve stop with savepoint ? - Milind On Mon, Apr 26, 2021 at 12:55 AM Matthias Pohl wrote: > I'm not sure what you're trying to achieve. Are you trying to simulate a > task failure? Or are you trying to pick up the state from a stopped job? > You could achieve the former

The heartbeat of JobManager timed out

2021-05-15 Thread Alexey Trenikhun
Hello, I periodically see in JM log (Flink 12.2): {"ts":"2021-05-15T21:10:36.325Z","message":"The heartbeat of JobManager with id be8225ebae1d6422b7f268c801044b05 timed out.","logger_name":"org.apache.flink.runtime.resourcemanager.StandaloneResourceManager","thread_name":"flink-akka.actor.defau