Here are some comments on Java Classloading: - The Zookeeper code is implicitly loaded by the Kafka code. - When java implicitly loads a class at some point in the program, is uses the classloader of the class of that point in the program. Here it will use the class loader that loaded the Kafka code. - You have to make sure that the zookeeper dependency is where the kafka connector is. It will not work to have kafka in /lib and zookeeper in the fat jar, for example.
But putting everything into the /lib folder should work. Note: Its important that this is available on all worker machines. On Wed, Jul 22, 2015 at 10:12 AM, Till Rohrmann <trohrm...@apache.org> wrote: > Hi Wendong, > > I don’t think that you have to include flink-streaming-connectors as a > dependency. In the 0.9.0-milestone-1 release, all connectors were still > bundled in this module. However, with the officiel 0.9.0 release, the > streaming connectors were split up in separate modules. > > Cheers, > Till > > > On Wed, Jul 22, 2015 at 9:59 AM, Hawin Jiang <hawin.ji...@gmail.com> > wrote: > >> Hi Wendong >> >> Please make sure you have dependencies as below. >> Good luck >> >> ********************************************************************* >> <dependency> >> <groupId>org.apache.flink</groupId> >> <artifactId>flink-java</artifactId> >> <version>0.9.0</version> >> </dependency> >> <dependency> >> <groupId>org.apache.flink</groupId> >> <artifactId>flink-clients</artifactId> >> <version>0.9.0</version> >> </dependency> >> <dependency> >> <groupId>org.apache.flink</groupId> >> <artifactId>flink-streaming-core</artifactId> >> <version>0.9.0</version> >> </dependency> >> <dependency> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>flink-streaming-connectors</artifactId> >> <version>0.9.0-milestone-1</version> >> </dependency> >> <dependency> >> <groupId>org.apache.kafka</groupId> >> <artifactId>kafka_2.10</artifactId> >> <version>0.8.2.1</version> >> </dependency> >> <dependency> >> <groupId>org.apache.kafka</groupId> >> <artifactId>kafka-clients</artifactId> >> <version>0.8.2.1</version> >> </dependency> >> ****************************************************************** >> >> >> >> >> Best regards >> Hawin >> >> -----Original Message----- >> From: Wendong [mailto:wendong....@gmail.com] >> Sent: Tuesday, July 21, 2015 5:16 PM >> To: user@flink.apache.org >> Subject: Re: Flink Kafka cannot find >> org/I0Itec/zkclient/serialize/ZkSerializer >> >> Hi Hawin, >> >> I'm using sbt as shown in the original post. I tried using maven and >> pom.xml, but got different NoClassDefFoundError: >> com/yammer/metrics/Metrics. >> I've downloaded metrics-core-2.2.0.jar under lib/ but it doesn't help. It >> seems the errors from sbt and Maven belong to same nature. Here is my >> pom.xml (standard parts are omitted to make it more readable): >> >> <dependencies> >> <dependency> >> <groupId>org.apache.flink</groupId> >> <artifactId>flink-scala</artifactId> >> <version>0.9.0</version> >> </dependency> >> <dependency> >> <groupId>org.apache.flink</groupId> >> <artifactId>flink-streaming-scala</artifactId> >> <version>0.9.0</version> >> </dependency> >> <dependency> >> <groupId>org.apache.flink</groupId> >> <artifactId>flink-clients</artifactId> >> <version>0.9.0</version> >> </dependency> >> <dependency> >> <groupId>org.apache.flink</groupId> >> <artifactId>flink-connector-kafka</artifactId> >> <version>0.9.0</version> >> </dependency> >> <dependency> >> <groupId>com.yammer.metrics</groupId> >> <artifactId>metrics-core</artifactId> >> <version>2.2.0</version> >> </dependency> >> </dependencies> >> >> <build> >> <plugins> >> >> >> >> >> >> <plugin> >> >> <groupId>org.apache.maven.plugins</groupId> >> >> <artifactId>maven-dependency-plugin</artifactId> >> <version>2.9</version> >> <executions> >> <execution> >> <id>unpack</id> >> >> <phase>prepare-package</phase> >> <goals> >> <goal>unpack</goal> >> </goals> >> <configuration> >> <artifactItems> >> >> <artifactItem> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>flink-connector-kafka</artifactId> >> >> <version>0.9.0</version> >> <type>jar</type> >> >> <overWrite>false</overWrite> >> >> <outputDirectory>${project.build.directory}/classes</outputDirectory> >> >> <includes>org/apache/flink/**</includes> >> </artifactItem> >> >> <artifactItem> >> >> <groupId>org.apache.kafka</groupId> >> >> <artifactId>kafka_2.10</artifactId> >> >> <version>0.8.2.0</version> >> <type>jar</type> >> >> <overWrite>false</overWrite> >> >> <outputDirectory>${project.build.directory}/classes</outputDirectory> >> >> <includes>kafka/**</includes> >> </artifactItem> >> </artifactItems> >> </configuration> >> </execution> >> </executions> >> </plugin> >> >> <plugin> >> >> <groupId>org.apache.maven.plugins</groupId> >> >> <artifactId>maven-compiler-plugin</artifactId> >> <version>3.1</version> >> <configuration> >> <source>1.6</source> >> <target>1.6</target> >> </configuration> >> </plugin> >> <plugin> >> <groupId>net.alchim31.maven</groupId> >> >> <artifactId>scala-maven-plugin</artifactId> >> <version>3.1.4</version> >> <executions> >> <execution> >> <goals> >> >> <goal>compile</goal> >> >> <goal>testCompile</goal> >> </goals> >> </execution> >> </executions> >> </plugin> >> >> >> >> </build> >> <profiles> >> <profile> >> >> <id>build-jar</id> >> <activation> >> <activeByDefault>false</activeByDefault> >> </activation> >> <dependencies> >> <dependency> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>flink-scala</artifactId> >> <version>0.9.0</version> >> <scope>provided</scope> >> </dependency> >> <dependency> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>flink-streaming-core</artifactId> >> <version>0.9.0</version> >> <scope>provided</scope> >> </dependency> >> <dependency> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>flink-clients</artifactId> >> <version>0.9.0</version> >> <scope>provided</scope> >> </dependency> >> </dependencies> >> </profile> >> </profiles> >> >> >> >> -- >> View this message in context: >> >> http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Ka >> >> fka-cannot-find-org-I0Itec-zkclient-serialize-ZkSerializer-tp2199p2222.html >> <http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Flink-Kafka-cannot-find-org-I0Itec-zkclient-serialize-ZkSerializer-tp2199p2222.html> >> Sent from the Apache Flink User Mailing List archive. mailing list archive >> at Nabble.com. >> >> >
