Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

Thanks for the example code! I'll try it out. Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-cannot-find-org-I0Itec-zkclient-serialize-ZkSerializer-tp2199p2294.html Sent from the Apache Flink User Mailing List ar

Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

Just to confirm: are you able to compile and run the program of testing Kafka similar to the following? import org.apache.kafka.clients.producer.{ProducerConfig, KafkaProducer, ProducerRecord} import org.apache.flink.streaming.api.environment._ import org.apache.flink.streaming.connectors.kafka

Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

Below is the build.sbt I am using (also include project/assembly.sbt) : //- Start build.sbt --- version := "1.0" scalaVersion := "2.10.4" libraryDependencies ++= Seq("org.apache.flink" % "flink-scala" % "0.9.0", "org.apache.flink" % "flink-clients" % "0.9.0")

Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

Spark sample, and it would greatly benefit the community. Wendong Here are some comments on Java Classloading: - The Zookeeper code is implicitly loaded by the Kafka code. - When java implicitly loads a class at some point in the program, is uses the classloader of the class of that point i

Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

Hi Hawin, I'm using sbt as shown in the original post. I tried using maven and pom.xml, but got different NoClassDefFoundError: com/yammer/metrics/Metrics. I've downloaded metrics-core-2.2.0.jar under lib/ but it doesn't help. It seems the errors from sbt and Maven belong to same nature. Here is m

Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

also tried using zkclient-0.3.jar in lib/, updated build.sbt and rebuild. It doesn't help. Still got the same error of NoClassDefFoundError: ZkSerializer in flink.streaming.connectors.kafka.api.KafkaSource.open(). -- View this message in context: http://apache-flink-user-mailing-list-archive.23

Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

Hi Max, The program runs locally in one machine. I use "grep ZkSerializer" in the generated fat jar file and it exists, so it seems build process is OK. I also put zkclient-0.5.jar under /lib/ and it contains the class of ZkSerializer. Thanks, Wendong -- View this message in cont

Re: Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

to be specific, the error occurs at: org.apache.flink.*streaming.connectors.kafka.api.KafkaSource.initializeConnection*(KafkaSource.java:175) -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-cannot-find-org-I0Itec-zkclient-seriali

Flink Kafka cannot find org/I0Itec/zkclient/serialize/ZkSerializer

.tasks.SourceStreamTask.invoke(SourceStreamTask.java:52) at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassNotFoundException: org.I0Itec.zkclient.serialize.ZkSerializer at java.net.URLClassLoader$1.run(U

Re: Flink Kafka example in Scala

Hi Till, Thanks for your suggestion! I did a fat jar and the runtime error of ClassNotFoundException was finally gone. I wish I had tried fat jar earlier and it would have saved me 4 days. Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4

Re: Flink Kafka example in Scala

so, I copied flink-connector-kafka-0.9.0.jar under /lib/, but there is still ClassNotFoundException for KafkaSink. I appreciate it if you have any suggestion. Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-example-in-Scala-t

Re: Flink Kafka example in Scala

java.net.URLClassLoader$1.run(URLClassLoader.java:366) Any clue? Thanks, Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-example-in-Scala-tp2069p2143.html Sent from the Apache Flink User Mailing List archive. mailing list archive at

Re: Flink Kafka runtime error

Hi Stephan, Thanks for your suggestion! I'll try the first solution too. Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-runtime-error-tp2063p2142.html Sent from the Apache Flink User Mailing List archive. mailing

Re: Flink Kafka example in Scala

"test", new SimpleStringSchema)) Anything wrong? I already did "import org.apache.flink.streaming.connectors.kafka.api._". Class SimpleStringSchema was modified (see previous post). Thanks, Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.

Re: Flink Kafka example in Scala

Thanks! I tried your updated MySimpleStringSchema and it works for both source and sink. However, my problem is the runtime error "Data stream sinks cannot be copied" as listed in previous post. I hope someone ran into the problem before and can give a hint. Wendong -- View this

Re: Flink Kafka example in Scala

org.apache.flink.streaming.api.datastream.DataStream.print(DataStream.java:1029) ... Googled the error message but didn't find useful information. Anyone can shed some light? Thanks! Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Flink Kafka example in Scala

trying to transit from Spark to Flink, but the samples in Flink are far less than those in Spark. It would be very helpful if there is an example of KafkaWordCount in Scala similar to that in Spark. Thanks, Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-example-in-Scala-tp2069.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Re: Flink Kafka runtime error

Just found a workaround. I downloaded kafka_2.10-0.8.2.1.jar and flink-connector-kafka-0.9.0.jar, then put them into $FLINK_HOME/lib/. Now the runtime error is gone. But this is just a workaound. I believe there is a better solution. Wendong -- View this message in context: http://apache

Flink Kafka runtime error

ka.api.KafkaSource at java.net.URLClassLoader$1.run(URLClassLoader.java:366) . Any additional setting is missing? Thanks for help! Wendong -- View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-runtime-error-tp2063.html Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.