Thanks for the explanation, that makes sense. For some reason I thought that in Yarn all stuff goes into the classpath.
Gy Chesnay Schepler <ches...@apache.org> ezt írta (időpont: 2018. júl. 11., Sze, 15:16): > Reporters do not have access to libraries provided with user-jars. > They are instantiated when JM/TM starts, i.e. before any user-code is > even accessible. > > My recommendation would be to either put the kafka dependencies in the > /lib folder or try to relocate the kafka code in the reporter. > > On 11.07.2018 14:59, Gyula Fóra wrote: > > Hi all, > > > > I have ran into the following problem and I want to double check > > wether this is intended behaviour. > > > > I have a custom metrics reporter that pushes things to Kafka (so it > > creates a KafkaProducer in the open method etc.etc.) for my streaming > job. > > > > Naturally as my Flink job consumes from Kafka so it has the kafka > > connector dependencies I set the Kafka dependencies to provided in my > > metric reporter project and I put the built kafkaReporter.jar into the > > Flink lib. However it seems that the metrics reporter is instantiated > > without the user code classes since I get a NoClassdefFound error for > > KafkaProducer even though my streaming job starts successfully > > reading/writing kafka. > > > > Any ideas why this happens and how to solve it? I am slightly against > > putting the kafka dependencies twice on the classpath as it has only > > caused problems in the past... > > > > Gyula > > >
