There is no way to reduce the number of metrics.
The only thing you can do is exclude specific variables (e.g.,
task_name), like this:
metrics.reporter.<reporter_name>.scope.variables.exclude:task_name[;
<any_other_variables_to_exclude]
On 12/9/2020 7:15 PM, Alexander Filipchik wrote:
Hi,
Is there a way to reduce cardinality (preaggregate) metrics that are
emitted to Prom Push gateway?
Our metrics infra is struggling to digest per task stats. Any way we
can configure it to emit per stage aggregates?
Our current config:
metrics.scope.tm <http://metrics.scope.tm> flink.taskmanager
metrics.scope.operator flink.operator
metrics.scope.jm <http://metrics.scope.jm> flink.jobmanager
metrics.scope.jm.job flink.jobmanager.job
metrics.scope.task flink.task
But metrics still look like:
{ job_id="b12e2",
job_name="kafka_",
subtask_index="2",
task_attempt_num="14",
task_id="00f9d",
task_name="Source:_Read_from_kafka",
tm_id="17e3c"
}
Am I changing a proper config?
Alex
