Hi Ruibin, could you switch to using the currently supported way for instantiating reporters using the factory configuration parameter [1][2]?
Based on the ClassNotFoundException, your suspicion might be right that the plugin didn't make it onto the classpath. Could you share the startup logs of the JM and TMs. That might help getting a bit more context on what's going on. Your approach on integrating the reporter through the plugin system [3] sounds about right as far as I can see. Matthias [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/#factory-class [2] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/#prometheus [3] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/filesystems/plugins/ On Fri, Dec 30, 2022 at 11:42 AM Ruibin Xing <xingro...@gmail.com> wrote: > Hi community, > > I am having difficulty understanding the Flink plugin system. I am > attempting to enable the Prometheus exporter with the official Flink image > 1.16.0, but I am experiencing issues with library dependencies. According > to the plugin documentation ( > https://nightlies.apache.org/flink/flink-docs-stable/docs/deployment/filesystems/plugins/), > as long as the library is located in the /opt/flink/plugins/<plugin-name> > directory, Flink should automatically load it, similar to how it loads > libraries in the /opt/flink/lib directory. However, Flink does not seem to > detect the plugin. > > Here is the directory structure for /opt/flink: > > tree /opt/flink > . > .... > ├── plugins > │ ├── metrics-prometheus > │ │ └── flink-metrics-prometheus-1.16.0.jar > ... > > And here is the related Flink configuration: > > metrics.reporter.prom.class: > org.apache.flink.metrics.prometheus.PrometheusReporter > > The error logs in the task manager show the following: > 2022-12-30 10:03:55,840 WARN > org.apache.flink.runtime.metrics.ReporterSetup [] - The > reporter configuration of 'prom' configures the reporter class, which is a > deprecated approach to configure reporters. Please configure a factory > class instead: 'metrics.reporter.prom.factory.class: <factoryClass>' to > ensure that the configuration continues to work with future versions. > 2022-12-30 10:03:55,841 ERROR > org.apache.flink.runtime.metrics.ReporterSetup [] - Could not > instantiate metrics reporter prom. Metrics might not be exposed/reported. > java.lang.ClassNotFoundException: > org.apache.flink.metrics.prometheus.PrometheusReporter > at jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source) ~[?:?] > at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown > Source) ~[?:?] > at java.lang.ClassLoader.loadClass(Unknown Source) ~[?:?] > at java.lang.Class.forName0(Native Method) ~[?:?] > at java.lang.Class.forName(Unknown Source) ~[?:?] > at > org.apache.flink.runtime.metrics.ReporterSetup.loadViaReflection(ReporterSetup.java:456) > ~[flink-dist-1.16.0.jar:1.16.0] > at > org.apache.flink.runtime.metrics.ReporterSetup.loadReporter(ReporterSetup.java:409) > ~[flink-dist-1.16.0.jar:1.16.0] > at > org.apache.flink.runtime.metrics.ReporterSetup.setupReporters(ReporterSetup.java:328) > ~[flink-dist-1.16.0.jar:1.16.0] > at > org.apache.flink.runtime.metrics.ReporterSetup.fromConfiguration(ReporterSetup.java:209) > ~[flink-dist-1.16.0.jar:1.16.0] > > The Java commands for Flink process: > flink 1 3.0 4.6 2168308 765936 ? Ssl 10:03 1:08 > /opt/java/openjdk/bin/java -XX:+UseG1GC -Xmx697932173 -Xms697932173 > -XX:MaxDirectMemorySize=300647712 -XX:MaxMetaspaceSize=268435456 > -Dlog.file=/opt/flink/log/flink--kubernetes-taskmanager-0-checkpoint-ha-example-taskmanager-1-1.log > -Dlog4j.configuration=file:/opt/flink/conf/log4j-console.properties > -Dlog4j.configurationFile=file:/opt/flink/conf/log4j-console.properties > -Dlogback.configurationFile=file:/opt/flink/conf/logback-console.xml > -classpath > /opt/flink/lib/flink-cep-1.16.0.jar:/opt/flink/lib/flink-connector-files-1.16.0.jar:/opt/flink/lib/flink-csv-1.16.0.jar:/opt/flink/lib/flink-json-1.16.0.jar:/opt/flink/lib/flink-scala_2.12-1.16.0.jar:/opt/flink/lib/flink-shaded-hadoop-2-uber-2.4.1-10.0.jar:/opt/flink/lib/flink-shaded-zookeeper-3.5.9.jar:/opt/flink/lib/flink-table-api-java-uber-1.16.0.jar:/opt/flink/lib/flink-table-planner-loader-1.16.0.jar:/opt/flink/lib/flink-table-runtime-1.16.0.jar:/opt/flink/lib/log4j-1.2-api-2.17.1.jar:/opt/flink/lib/log4j-api-2.17.1.jar:/opt/flink/lib/log4j-core-2.17.1.jar:/opt/flink/lib/log4j-slf4j-impl-2.17.1.jar:/opt/flink/lib/flink-dist-1.16.0.jar:::: > org.apache.flink.kubernetes.taskmanager.KubernetesTaskExecutorRunner > --configDir /opt/flink/conf -Djobmanager.rpc.address=172.17.0.7 > -Dpipeline.classpaths= -Djobmanager.memory.off-heap.size=134217728b > -Dweb.tmpdir=/tmp/flink-web-57b9e638-f313-4389-a75b-988509697edb > -Djobmanager.rpc.port=6123 > -D.pipeline.job-id=ffffffffa6f1c9fb0000000000000000 > -Drest.address=172.17.0.7 -Djobmanager.memory.jvm-overhead.max=214748368b > -Djobmanager.memory.jvm-overhead.min=214748368b > -Dtaskmanager.resource-id=checkpoint-ha-example-taskmanager-1-1 > -Dexecution.target=embedded > -Dpipeline.jars=file:/opt/flink/examples/streaming/StateMachineExample.jar > -Djobmanager.memory.jvm-metaspace.size=268435456b > -Djobmanager.memory.heap.size=1530082096b -D > taskmanager.memory.network.min=166429984b -D taskmanager.cpu.cores=1.0 -D > taskmanager.memory.task.off-heap.size=0b -D > taskmanager.memory.jvm-metaspace.size=268435456b -D external-resources=none > -D taskmanager.memory.jvm-overhead.min=214748368b -D > taskmanager.memory.framework.off-heap.size=134217728b -D > taskmanager.memory.network.max=166429984b -D > taskmanager.memory.framework.heap.size=134217728b -D > taskmanager.memory.managed.size=665719939b -D > taskmanager.memory.task.heap.size=563714445b -D > taskmanager.numberOfTaskSlots=1 -D > taskmanager.memory.jvm-overhead.max=214748368b > > It seems that I must be missing something. Could someone please help me > clarify this issue? > > Thank you, > > Ruibin >
