Re: yarnship option

2017-05-23 Thread Mikhail Pryakhin
Hi Robert, Thanks for your advice but this option doesn’t work for us because our job depends on third party jars with transitive dependencies to the same jar but of different versions and it hardly possible to say which version will participate in the fat jar.. Anyway thanks a lot for your he

Re: yarnship option

2017-05-23 Thread Robert Metzger
Hi Mike, I would recommend you to build a "fat jar" containing your application code and all required dependencies. On Tue, May 23, 2017 at 10:33 AM, Mikhail Pryakhin wrote: > Hi Robert! > > You were right.. it was my fault, I didn’t copy all required dependencies > for the flink-connector-kafk

Re: yarnship option

2017-05-23 Thread Mikhail Pryakhin
Hi Robert! You were right.. it was my fault, I didn’t copy all required dependencies for the flink-connector-kafka. That made impossible to load the class. Everything works as expected but only if I put required dependencies inside the flink lib folder, but when dependencies are located in a di

Re: yarnship option

2017-05-22 Thread Mikhail Pryakhin
Hi Robert!Thanks a lot for your reply!>Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ?The jar does contain the class org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09.class (The

Re: yarnship option

2017-05-22 Thread Robert Metzger
Hi, this issue is unexpected :) Can you double check if the job-libs/flink-connector-kafka-0.9_2.11-1.2.1.jar contains org/apache/flink/streaming/connectors/kafka/FlinkKafkaConsumer09 ? Also, were there any previous Kafka09 related exceptions in the log?? >From this SO answer, it seems that this