Different jobName per Job when reporting Flink metrics to PushGateway

[Sidney Feiner]

I'm using Flink 1.9.1 with PrometheusPushGateway to report my metrics. The 
jobName the metrics are reported with is defined in the flink-conf.yaml file 
which makes the jobName identical for all jobs who run on the cluster, but I 
want a different jobName to be reported for every running job. To do so, I 
tried doing the following in my code before executing the Stream:

Configuration conf = GlobalConfiguration.loadConfiguration();
    conf.setString(
            "metrics.reporter.promgateway.jobName",
            conf.getString("metrics.reporter.promgateway.jobName", "") + "-" + 
pipeline
    );
    final StreamExecutionEnvironment env = 
StreamExecutionEnvironment.getExecutionEnvironment();
    env.getConfig().setGlobalJobParameters(conf);


When pipeline is a String variable.

When running the job locally, it worked. But now I'm running flink in High 
Availability mode and it doesn't work anymore :( The config I override in the 
code is ignored.

So how can I change the jobName per job? And if I can't, is there a way to set 
additional Labels when reporting the metrics? Because I haven't seen an option 
for that as well.

Thanks :)


I've posted this on StackOverflow as well - 
here<https://stackoverflow.com/questions/59376693/different-jobname-per-job-when-reporting-flink-metrics-to-pushgateway>
 :)



Sidney Feiner / Data Platform Developer
M: +972.528197720 / Skype: sidney.feiner.startapp

[emailsignature]