Prometheus labels in metrics / counters

[Witold Baryluk]

Hi,

I want to track and increment some monitoring counters from `map`,
but have them broken down by dynamically defined values of a label.
The set of values is unknown at creation time, but it is bounded
(less than 100 different values during a 30 day period, usually ~5).

There is no easy way of doing this, compared to other Prometheus
native systems (i.e. in Go, or C++), but it looks like it might be possible
using some workarounds:


public class MyMapper extends RichMapFunction<String, String> {
  private transient MetricGroup metric_group;

  @Override
  public void open(Configuration config) {
    this.metric_group = getRuntimeContext().getMetricGroup();
  }

  @Override
  public String map(String value) throws Exception {
      Group group = this.metric_group.addGroup("kind", getKind(value));
      group.counter("latency_sum").inc(getLatencyMicros(value));
      group.counter("latency_count").inc();
      return value;
  }
}

Will this work? Is there a better way?

But, this does not look nice at all compared to how
other projects handle Prometheus labels. Flink metrics are not well
mapped into Prometheus metrics here.

Second question, is it ok to call addGroup and counter, dynamically like this,
or should it be cached? Do I need any such cache (which would be map
of string to Counter), be protected by some mutex when I lookup or add
to it?

Cheers,
Witold