Hi, this issue is unexpected :) Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ? Also, were there any previous Kafka09 related exceptions in the log??
>From this SO answer, it seems that this is not really the classical classNotFoundException, but a bit differenT: https://stackoverflow.com/a/5756989/568695 On Mon, May 22, 2017 at 5:12 PM, Mikhail Pryakhin <m.prya...@cleverdata.ru> wrote: > Hi all! > > I'm playing with flink streaming job on yarn cluster. The job consumes > events from kafka and prints them to the standard out. > The job uses flink-connector-kafka-0.9_2.11-1.2.1.jar library that is > passed via the --yarnship option. > Here is the way I run the job: > > export HADOOP_USER_NAME=hdfs; \ > export HADOOP_CONF_DIR=/etc/hadoop/conf/; \ > /opt/flink-1.2.1/bin/flink run \ > -yst \ > -yt /home/user/job-libs \ > -m yarn-cluster \ > -yn 3 \ > -c com.flink.Test \ > flink-test_2.11-1.0.0-SNAPSHOT.jar > > Finally the job fails complaing that it can't find the class: > java.lang.NoClassDefFoundError: org/apache/flink/streaming/ > connectors/kafka/FlinkKafkaConsumer09 > > I looked through the jobmanager.log and found that the > flink-connector-kafka-0.9_2.11-1.2.1.jar library is added to the > classpath: > > 2017-05-22 17:41:59,637 INFO > org.apache.flink.yarn.YarnApplicationMasterRunner > - Classpath: job-libs/flink-connector- > kafka-0.9_2.11-1.2.1.jar:<another jars...> > > Could please help to figure out why the class org/apache/flink/streaming/ > connectors/kafka/FlinkKafkaConsumer09 can't be loaded inspite of the jar > containing this class was added to the classpath > > ---- > Mike Pryakhin
