We are running a Flink 1.11.0 job cluster on Kubernetes. We're not seeing
any metrics in the Flink Web UI (for the default metrics like Bytes
Received, Records Received, etc.), instead we see a spinner. See image
below.
However, we have a prometheus metrics exporter configured and see job/task
metrics in Prometheus.
[image: Flink Web UI Spinner.png]
Looking into the network tab, for requests that retrieve those metrics, we
see
GET
/jobs/296402a32a8bbb1917279b5c2b2f40f1/vertices/cbc357ccb763df2852fee8c4fc7d55f2/watermarks
The response is an empty array
if I do this via CURL
GET
/jobs/296402a32a8bbb1917279b5c2b2f40f1/vertices/cbc357ccb763df2852fee8c4fc7d55f2
I get back
{
"id": "cbc357ccb763df2852fee8c4fc7d55f2",
"name": "Source: Custom Source -> Timestamps/Watermarks -> Filter",
"now": 1599776105604,
"parallelism": 1,
"subtasks": Array[1][
{
"attempt": 0,
"duration": 197613130,
"end-time": -1,
"host": "100.96.18.38",
"metrics": {
"read-bytes": 0,
"read-bytes-complete": false,
"read-records": 0,
"read-records-complete": false,
"write-bytes": 0,
"write-bytes-complete": false,
"write-records": 0,
"write-records-complete": false
},
"start-time": 1599578492474,
"start_time": 1599578492474,
"status": "RUNNING",
"subtask": 0,
"taskmanager-id": "2eb0550f0d3bca170f76dd86f84843a0"
}
]
}
Are we missing anything in our setup? Any insight would be greatly
appreciated.
Thanks
