It is good to know that something from the task executors arrives at
datadog.
Do you see any metrics for a specific job, like the numRestarts metric
of the JobManager?
Are you using the default scope formats
<https://ci.apache.org/projects/flink/flink-docs-release-1.12/ops/metrics.html#system-scope>,
or have you modified them?
Could you try these instead and report back? (I replaced all
job/task/operator names with their IDs, in case some special character
is messing with datadog)
metrics.scope.jm: <host>.jobmanager
metrics.scope.jm.job: <host>.jobmanager.<job_id>
metrics.scope.tm: <host>.taskmanager.<tm_id>
metrics.scope.tm.job: <host>.taskmanager.<tm_id>.<job_id>
metrics.scope.task:
<host>.taskmanager.<tm_id>.<job_id>.<task_id>.<subtask_index>
metrics.scope.operator:
<host>.taskmanager.<tm_id>.<job_id>.<operator_id>.<subtask_index>
On 1/26/2021 9:28 PM, Rex Fenley wrote:
All taskmanager and jobmanager logs show up. Anything specific to an
operator does not.
For example, flink.taskmanager.Status.JVM.Memory.Heap.Used shows up,
but I can't see stats on an individual operator.
I mostly followed a combination of
https://docs.datadoghq.com/integrations/flink/#metric-collection
<https://docs.datadoghq.com/integrations/flink/#metric-collection> and
https://ci.apache.org/projects/flink/flink-docs-release-1.11/monitoring/metrics.html#datadog-orgapacheflinkmetricsdatadogdatadoghttpreporter
<https://ci.apache.org/projects/flink/flink-docs-release-1.11/monitoring/metrics.html#datadog-orgapacheflinkmetricsdatadogdatadoghttpreporter>
since datadog's documentation was slightly out of date.
Thanks
On Tue, Jan 26, 2021 at 10:28 AM Chesnay Schepler <ches...@apache.org
<mailto:ches...@apache.org>> wrote:
Anything metric that is shown in the Flink UI should also appear
in DataDog.
If this is not the case then something goes wrong within the reporter.
Is there anything suspicious in the Flink logs?
Can you give some example of metrics that /do/ show up in DataDog?
On 1/26/2021 6:32 PM, Rex Fenley wrote:
Hi,
I need to get a deeper dive into how rocks is performing so I
turned on Rocks Native Metrics. However, I don't see any of the
metrics in DataDog (though I have other Flink metrics in
DataDog). I only see rocks metrics in the operator metrics in
Flink UI, and unfortunately I can't really zoom in or out of
those metrics or compare against multiple operators at a time
which makes it really difficult to get an overview of how rocks
is doing.
Is this there any way to get the Rocks Native Metrics forwarded
over to DataDog?
Thanks!
--
Rex Fenley|Software Engineer - Mobile and Backend
Remind.com <https://www.remind.com/>| BLOG
<http://blog.remind.com/> | FOLLOW US
<https://twitter.com/remindhq> | LIKE US
<https://www.facebook.com/remindhq>
--
Rex Fenley|Software Engineer - Mobile and Backend
Remind.com <https://www.remind.com/>| BLOG <http://blog.remind.com/> |
FOLLOW US <https://twitter.com/remindhq> | LIKE US
<https://www.facebook.com/remindhq>
