Now I see what you mean. I think you would have to somehow set up the
Flink metrics system as a backend for opencensus. Then the metrics would
be reported to the same system (prometheus) in this case. In Opencensus
lingo, this would mean using a Flink-based Stats Exporter instead of the
Prometheus backend directly.
I don't have experience developing opencensus Stats Exporters but
working off the provided ones should work. Maybe have a look at the
prometheus exporter [1] and go from there. The entrypoint for a Flink
exporter would then probably be
FlinkStatsCollector.createAndRegister(getMetricGroup());
Best,
Aljoscha
On 06.05.20 17:11, Eleanore Jin wrote:
Hi Aljoscha,
Thanks for the response, yes prometheus reporter is already enabled and I
am able to get the flink metrics from prometheus.
My question is more like, currently I am using opencensus library
<https://github.com/census-instrumentation/opencensus-java> to collect
application metrics, so is there an easy way to integrate this metrics with
flink metrics endpoint.
Thanks!
Eleanore
On Wed, May 6, 2020 at 7:48 AM Aljoscha Krettek <aljos...@apache.org> wrote:
Hi,
that should be possible. Did you have a look at the documentation for
setting up a Prometheus metrics reporter:
https://ci.apache.org/projects/flink/flink-docs-master/monitoring/metrics.html#prometheus-orgapacheflinkmetricsprometheusprometheusreporter
Best,
Aljoscha
On 06.05.20 03:02, Eleanore Jin wrote:
Hi all,
I just wonder is it possible to use Flink Metrics endpoint to allow
Prometheus to scrape user defined metrics?
Context:
In addition to Flink metrics, we also collect some application level
metrics using opencensus. And we run opencensus agent as side car in
kubernetes pod to collect metrics (opencensus agent talks to task manager
container via rpcs)
The issue with this approach is: it looks like opencensus agent keeps
staled metrics, causing the metrics reporting inaccurate, and this
project
is not actively maintained anymore.
So I wonder if it is possible to use Flink metrics endpoint for user
defined metrics.
Thanks a lot!
Eleanore