Re: Capturing statistics for more than 5 minutes

Hi I have been able to run JMX with Flink with the following configuration applied to the flink-conf.yaml file of all nodes in the cluster: metrics.reporters: jmx metrics.reporter.jmx.class: org.apache.flink.metrics.jmx.JMXReporter metrics.reporter.jmx.port: 9020-9022 env.java.opts: -Dcom.sun.ma

Re: Apache Flink - Evictor interface clarification

Hi Mans, Have a look at this: http://apache-flink-mailing-list-archive.1008284. n3.nabble.com/DISCUSS-Enhance-Window-Evictor-in-Flink-tp12406p12442.html Thanks, Vishnu On Sat, Nov 18, 2017 at 11:28 AM, M Singh wrote: > Hi: > > I am looking at the documentation and

RE: Job Manager Configuration

Is your job running on a standalone cluster? I’m using a detached yarn session in a multi-tenant environment. And I’m guessing you haven’t had to do anything special for the akka configurations. From: Joshua Griffith [mailto:jgriff...@campuslabs.com] Sent: Thursday, November 16, 2017 2:57 PM To

Apache Flink - Evictor interface clarification

Hi: I am looking at the documentation and it states: 'The evictor has the ability to remove elements from a window after the trigger fires and before and/or after the window function is applied. ' I understand that if we use evictor's before method (evictBefore) it can remove items before invocat

Re: all task managers reading from all kafka partitions

Gary, thanks a lot! I completely forgot that parallelism extends over all slots visible to the JobManager! So adding e.g. -p4 to 'flink run' approach should suit my use case just fine, I believe. I'll look deeper into failure recovery with this scheme Have a great weekend! -Robert >

Re: all task managers reading from all kafka partitions

Hi Robert, Running a single job does not mean that you are limited to a single JVM. For example, a job with parallelism 4 by default requires 4 task slots to run. You can provision 4 single slot TaskMangers on different hosts to connect to the same JobManager. The JobManager can then take your jo