Thank you for replying!
Will exclusion produce proper aggregates? If I drop subtask_index, will
resulting metric be a sum of all the subtasks, or it will be just data from
one that was reported the last?
Alex
On Thu, Dec 10, 2020 at 4:28 AM Chesnay Schepler <ches...@apache.org> wrote:
> There is no way to reduce the number of metrics.
>
> The only thing you can do is exclude specific variables (e.g., task_name),
> like this:
> metrics.reporter.<reporter_name>.scope.variables.exclude: task_name[;
> <any_other_variables_to_exclude]
>
> On 12/9/2020 7:15 PM, Alexander Filipchik wrote:
>
> Hi,
>
> Is there a way to reduce cardinality (preaggregate) metrics that are
> emitted to Prom Push gateway?
>
> Our metrics infra is struggling to digest per task stats. Any way we can
> configure it to emit per stage aggregates?
>
> Our current config:
> metrics.scope.tm flink.taskmanager
> metrics.scope.operator flink.operator
> metrics.scope.jm flink.jobmanager
> metrics.scope.jm.job flink.jobmanager.job
> metrics.scope.task flink.task
>
> But metrics still look like:
> { job_id="b12e2",
> job_name="kafka_",
> subtask_index="2",
> task_attempt_num="14",
> task_id="00f9d",
> task_name="Source:_Read_from_kafka",
> tm_id="17e3c"
> }
>
> Am I changing a proper config?
>
> Alex
>
>
>
