Re: MetricRegistryTestUtils java class (flink-runtime/metrics) not found in source code version 1.14.3

Hi Prasanna, Why do you need histograms vs summaries? I'm curious about the change and want to see if it applies to my usage of the PrometheusReporter. Best, Mason On Mon, Jan 31, 2022 at 11:51 PM Martijn Visser wrote: > Hi Prasanna, > > Just a quick note that the Github links are all pointing

Re: Flink job recovery after task manager failure

Hi Zhilong, I will check the issues you raised. Thanks for your help, Ifat From: Zhilong Hong Date: Thursday, 24 February 2022 at 19:58 To: "Afek, Ifat (Nokia - IL/Kfar Sava)" Cc: "user@flink.apache.org" Subject: Re: Flink job recovery after task manager failure Hi, Afek I've read the log y

Re: Low Watermark

Hi, It's the minimal watermark among all 10 parallel instances of that Task. Using metric (currentInputWatermark) [1] you can access the watermark of each of those 10 sub tasks individually. Best, Piotrek [1] https://nightlies.apache.org/flink/flink-docs-master/docs/ops/metrics/ pt., 25 lut 20

Re: Possible BUG in 1.15 SQL JSON_OBJECT()

Thank you for reporting! That is definitely a bug, and I have opened a ticket to fix which you can track here. https://issues.apache.org/jira/browse/FLINK-26374 Seth On Thu, Feb 24, 2022 at 4:18 PM Jonathan Weaver wrote: > Using the latest SNAPSHOT BUILD. > > If I have a column definition as >

Low Watermark

Hello, could someone please explain what the Low Watermark indicates in the Flink UI in the attached image? I have event time enabled with a boundOutOfOrdernessStrategy of 3s for the incoming events and I use CEP with a within window of 5 minutes.

Pods are OOMKilled with RocksDB backend after a few checkpoints

Hi all, I am trying to reduce the memory usage of a Flink app. There is about 25+Gb of state when persisted to checkpoint/savepoint. And a fair amount of short lived objects as incoming traffic is fairly high. So far, I have 8TM with 20GB each using Flink 1.12. I would like to reduce the amount of

source code build failure

Hi all, I was trying to build flink source code by using mvn clean install -DskipTests command, and it turned out report build failure and the following error info: [ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile (default-testCompile) on projec

Re: streaming mode with both finite and infinite input sources

This should be supported in 1.14 if you enable checkpointing with finished tasks[1], which has been added in 1.14. In 1.15 it will be enabled by default. Best, Dawid [1] https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/checkpointing/#enabling-and-confi

Re: [Flink-1.14.3] Restart of pod due to duplicatejob submission

Thank you Yang for your inputs...! As a workaround I am doing the same by clearing up its state from ZK. I am also evaluating alternatives for this by removing ZK based HA. As our application is already in k8s, and pods are resilient. Also, AFAIK, flink does not support active-active HA. So it wo

Re: streaming mode with both finite and infinite input sources

One possible option is to look into the hybrid source released in Flink 1.14 to support your use-case: https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/connectors/datastream/hybridsource/ On Fri, Feb 25, 2022, 09:19 Jin Yi wrote: > so we have a streaming job where the main work t