Hi, I need to capture the Metrics sent from a Flink app to a Reporter and transform them to an Events API format I have designed. I have been looking at the Reporters( https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/metrics.html#list-of-all-variables) and have used them but what would be a best practice to capture this metrics data to transform it ?
The folks using the Flink app still want to see their metrics in the Flink Dashboard using their chosen(not sure yet what they chose-assuming ConsoleReporter) Reporter. I need to capture those metrics, transform them to my Events API format and send it to a Kinesis Stream. We use Prometheus and InfluxDB in our environments for other purposes. Should I use the SLF4J Reporter to dump the metrics into a log file/folder and watch that with a Kinesis Agent and transform it somehow(?) and then send it to the Kinesis data stream ? TIA,
