Re: Apache Flink - Question about metric registry and reporter and context information

2019-06-15 Thread Yun Tang
Hi 1) Yes, the metrics reporter was instantiated per task manager, you could refer to [1] to confirm. 2) You could get your runtime context by calling #getRuntimeContext() in RichFunction. And you could get your metric group within runtimeContext then. The task manager name could be found by #

Best practice to process DB stored log (is Flink the right choice?)

2019-06-15 Thread Stefano Lissa
Hi, surely really new-bye question but Flink sounds like to be the best choice. I have a log continuously added to a database table where a machine status is stored with a timestamp (the status is 0, 1, 2). What I need is to process this status and produce another stream where a sequence of status

Re: Controlling the amount of checkpoint files

2019-06-15 Thread Boris Lublinsky
So if you have externalized checkpoints, they are never purged? The issue is that if your state size is rather large, this seems to be the only option. Boris Lublinsky FDP Architect boris.lublin...@lightbend.com https://www.lightbend.com/ > On Jun 15, 2019, at 3:39 AM, Robert Metzger wrote: >

Timeout about local test case

2019-06-15 Thread aitozi
Hi community, When I run mvn clean install locally (branch master) the test case always failed with bundle timeout exception, I am doubt why this failed? And also an ip PS: I run the case individually also met the same error.

Re: Controlling the amount of checkpoint files

2019-06-15 Thread Robert Metzger
Hey Boris, I think the problem is that you are using externalized checkpoints: env.getCheckpointConfig.enableExternalizedCheckpoints(ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION) Your checkpoints are retained in both failure and cancellation cases, so the checkpoint files with grow indef