Reporters do not have access to libraries provided with user-jars.
They are instantiated when JM/TM starts, i.e. before any user-code is
even accessible.
My recommendation would be to either put the kafka dependencies in the
/lib folder or try to relocate the kafka code in the reporter.
On 11.07.2018 14:59, Gyula Fóra wrote:
Hi all,
I have ran into the following problem and I want to double check
wether this is intended behaviour.
I have a custom metrics reporter that pushes things to Kafka (so it
creates a KafkaProducer in the open method etc.etc.) for my streaming job.
Naturally as my Flink job consumes from Kafka so it has the kafka
connector dependencies I set the Kafka dependencies to provided in my
metric reporter project and I put the built kafkaReporter.jar into the
Flink lib. However it seems that the metrics reporter is instantiated
without the user code classes since I get a NoClassdefFound error for
KafkaProducer even though my streaming job starts successfully
reading/writing kafka.
Any ideas why this happens and how to solve it? I am slightly against
putting the kafka dependencies twice on the classpath as it has only
caused problems in the past...
Gyula
