Pankaj, I just checked, and the latency metrics for SocketWindowWordCount show up just fine for me with Flink 1.11.1. For me, the latency metrics existed even before I provided any data on the socket for the job to process. This makes sense, as the latency tracking markers will propagate through the job graph and provide measurements despite the lack of any real data for the job to process.
These metrics have horrible names, but for me, curl -s http://localhost:8081/jobs/69f954cb93c680fb2b210f877c377131/metrics reveals a bunch of latency metrics, and, for example, curl -s http://localhost:8081/jobs/69f954cb93c680fb2b210f877c377131/metrics?get=latency.source_id.cbc357ccb763df2852fee8c4fc7d55f2.operator_id.17fbfcaabad45985bbdf4da0490487e3.operator_subtask_index.0.latency_p999 returns [{"id":"latency.source_id.cbc357ccb763df2852fee8c4fc7d55f2.operator_id.17fbfcaabad45985bbdf4da0490487e3.operator_subtask_index.0.latency_p999","value":"105.0"}] Best, David On Wed, Sep 9, 2020 at 3:16 PM Pankaj Chand <pankajchanda...@gmail.com> wrote: > Hi David, > > Thanks for replying! Sorry, I forgot to mention I am using Flink Version: > 1.11.1, Commit ID: 7eb514a. > Is it possible that the default SocketWindowWordCount job is too simple to > generate Latency metrics? Or that the latency metrics disappear from the > output JSON when the data ingestion is zero? > > Thanks, > > Pankaj > > > On Wed, Sep 9, 2020 at 6:27 AM David Anderson <da...@alpinegizmo.com> > wrote: > >> Pankaj, >> >> The Flink web UI doesn't do any visualizations of histogram metrics, so >> the only way to access the latency metrics is either through the REST api >> or a metrics reporter. >> >> The REST endpoint you tried is the correct place to find these metrics in >> all recent versions of Flink, but somewhere back before Flink 1.5 or 1.6 >> (if I recall correctly) these metrics were task metrics. So if you are >> using an older version of Flink you'll need to dig deeper. I believe you'll >> find them in >> >> /jobs/<jobid>/vertices/<vertexid>/subtasks/metrics >> >> Regards, >> David >> >> >> >> On Tue, Sep 8, 2020 at 10:52 PM Pankaj Chand <pankajchanda...@gmail.com> >> wrote: >> >>> Hello, >>> >>> How do I visualize (or extract) the results for Latency Tracking for a >>> Flink local cluster? I set "metrics.latency.interval 100" in the >>> conf/flink-conf.yaml file, and started the cluster and >>> SocketWindowWordCount job. However, I could not find the latency >>> distributions anywhere in the web UI, nor are there any latency metrics in >>> the Metrics dropdown box for either task. >>> >>> I also set "metrics.latency.granularity "operator"" in >>> conf/flink-conf.yaml, but that did not help. >>> >>> When I tried to query the REST endpoint, I got the following output >>> which did not seem to contain anything related to latency: >>> >>> $ curl -s >>> http://localhost:8081/jobs/5d0e348eb68588646dece3654d846cf3/metrics >>> >>> >>> [{"id":"numberOfFailedCheckpoints"},{"id":"lastCheckpointSize"},{"id":"lastCheckpointExternalPath"},{"id":"totalNumberOfCheckpoints"},{"id":"lastCheckpointRestoreTimestamp"},{"id":"uptime"},{"id":"restartingTime"},{"id":"numberOfInProgressCheckpoints"},{"id":"downtime"},{"id":"numberOfCompletedCheckpoints"},{"id":"numRestarts"},{"id":"fullRestarts"},{"id":"lastCheckpointDuration"}] >>> >>> Thanks, >>> >>> Pankaj >>> >>
