Hi Timo,
Thank you for the reply. Not really sure how that link helps besides explaining what a histogram is or accessing the metrics through the UI which is not what im interested in. With flink 1.12 and 1.13 the latency metric was working great, however, with 1.14 and the new KafkaSource/KafkaSink these metrics are just not being generated. Can we confirm that it has been implemented? Regards, Morgan. ________________________________ From: Timo Walther <twal...@apache.org> Sent: 13 December 2021 09:53:08 To: user@flink.apache.org Subject: Re: Latency monitoring in Flink 1.14.0 Hi Morgan, did you see this: https://stackguides.com/questions/68917956/read-flink-latency-tracking-metric-in-datadog Also `metrics.latency.granularity` must be set in the Flink configuration. Not sure if `-D` forwards this properly. Timo On 10.12.21 18:31, Geldenhuys, Morgan Karl wrote: > Greetings all, > > > I am attempting to setup latency monitoring for a flink 1.14.0 job. > According to the documentation > <https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/ops/metrics/>, > I have done the following: > > > In my kubernetes setup I have added the following to the > kubernetes-session.sh command: > > -Dmetrics.latency.granularity=\"operator\" \ > -Dmetrics.latency.interval=1000 \ > > However, when looking at Prometheus, I do not see histograms related to > latency (prometheus is configured correctly). > > > I have added the following to my job pom and am using the new > KafkaSource/KafkaSink classes: > > > <dependency> > <groupId>org.apache.flink</groupId> > <artifactId>flink-streaming-java_2.11</artifactId> > <version>1.14.0</version> > </dependency> > <dependency> > <groupId>org.apache.flink</groupId> > <artifactId>flink-clients_2.11</artifactId> > <version>1.14.0</version> > </dependency> > <dependency> > <groupId>org.apache.flink</groupId> > <artifactId>flink-connector-kafka_2.11</artifactId> > <version>1.14.0</version> > </dependency> > <dependency> > <groupId>org.apache.flink</groupId> > <artifactId>flink-statebackend-rocksdb_2.11</artifactId> > <version>1.14.0</version> > </dependency> > > > Would really appreciate some help here. Thanks in advance! > > > Regards, > > Morgan. > > > > >
