Hi Kai, You can use the excluded variables config for the reporter. metrics.reporter.<name>.scope.variables.excludes: (optional) A semi-colon (;) separate list of variables that should be ignored by tag-based reporters (e.g., Prometheus, InfluxDB).
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/#reporter <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/#reporter> Best, Mason > On Jun 3, 2021, at 9:31 PM, Kai Fu <zzfu...@gmail.com> wrote: > > Hi team, > > We noticed that Prometheus metrics exporter exports all of the metrics at the > most fine-grained level, which is tremendous for the prometheus server > especially when the parallelism is high. The metrics volume crawled from a > single host(parallelism 8) is around 40MB for us currently. This is due to > task_name attribute in the metrics generated by the engine being very long. > The task_name attribute is auto-generated from SQL job, and it seems it's > attaching all field names onto it. > > We want to reduce the metrics volume by either drop task_name or at some more > coarse-grained level. But I cannot find any related documents about this, any > advice on that? > > [1] > https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/ > > <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/deployment/metric_reporters/> > [2] > https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#list-of-all-variables > > <https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/ops/metrics/#list-of-all-variables> > > -- > Best wishes, > - Kai
