Robert,
That did not fix it ( using flink and connector same version) . Tried
with scala version 2.11, so will try to see scala 2.10 makes any
difference.
balaji
On Fri, Mar 11, 2016 at 8:06 PM, Robert Metzger <rmetz...@apache.org> wrote:
> Hi,
>
> you have to use the same version for all dependencies from the
> "org.apache.flink" group.
>
> You said these are the versions you are using:
>
> flink.version = 0.10.2
> kafka.verison = 0.8.2
> flink.kafka.connection.verion=0.9.1
>
> For the connector, you also need to use 0.10.2.
>
>
>
> On Fri, Mar 11, 2016 at 9:56 AM, Balaji Rajagopalan <
> balaji.rajagopa...@olacabs.com> wrote:
>
>> I am tyring to use the flink kafka connector, for this I have specified
>> the kafka connector dependency and created a fat jar since default flink
>> installation does not contain kafka connector jars. I have made sure that
>> flink-streaming-demo-0.1.jar has the
>> kafka.javaapi.consumer.SimpleConsumer.class but still I see the class not
>> found exception.
>>
>> The code for kafka connector in flink.
>>
>> val env = StreamExecutionEnvironment.getExecutionEnvironment
>> val prop:Properties = new Properties()
>> prop.setProperty("zookeeper.connect","somezookeer:2181")
>> prop.setProperty("group.id","some-group")
>> prop.setProperty("bootstrap.servers","somebroker:9092")
>>
>> val stream = env
>> .addSource(new FlinkKafkaConsumer082[String]("location", new
>> SimpleStringSchema, prop))
>>
>> jar tvf flink-streaming-demo-0.1.jar | grep
>> kafka.javaapi.consumer.SimpleConsumer
>>
>> 5111 Fri Mar 11 14:18:36 UTC 2016
>> *kafka/javaapi/consumer/SimpleConsumer*.class
>>
>> flink.version = 0.10.2
>> kafka.verison = 0.8.2
>> flink.kafka.connection.verion=0.9.1
>>
>> The command that I use to run the flink program in yarn cluster is below,
>>
>> HADOOP_CONF_DIR=/etc/hadoop/conf /usr/share/flink/bin/flink run -c
>> com.dataartisans.flink_demo.examples.DriverEventConsumer -m yarn-cluster
>> -yn 2 /home/balajirajagopalan/flink-streaming-demo-0.1.jar
>>
>> java.lang.NoClassDefFoundError: kafka/javaapi/consumer/SimpleConsumer
>>
>> at
>> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.getPartitionsForTopic(FlinkKafkaConsumer.java:691)
>>
>> at
>> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer.<init>(FlinkKafkaConsumer.java:281)
>>
>> at
>> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer082.<init>(FlinkKafkaConsumer082.java:49)
>>
>> at
>> com.dataartisans.flink_demo.examples.DriverEventConsumer$.main(DriverEventConsumer.scala:53)
>>
>> at
>> com.dataartisans.flink_demo.examples.DriverEventConsumer.main(DriverEventConsumer.scala)
>>
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>
>> at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>
>> at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>
>> at java.lang.reflect.Method.invoke(Method.java:497)
>>
>> at
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:497)
>>
>> at
>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:395)
>>
>> at org.apache.flink.client.program.Client.runBlocking(Client.java:252)
>>
>> at
>> org.apache.flink.client.CliFrontend.executeProgramBlocking(CliFrontend.java:676)
>>
>> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:326)
>>
>> at
>> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:978)
>>
>> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1028)
>>
>> Caused by: java.lang.ClassNotFoundException:
>> kafka.javaapi.consumer.SimpleConsumer
>>
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
>>
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>
>> ... 16 more
>>
>>
>> Any help appreciated.
>>
>>
>> balaji
>>
>
>
