Dear Marton What do you meaning for locally Eclipse with 'Run'. Do you want to me to run it on Namenode? But my namenode didn't install Kafka. I only installed Kafka on my data node servers. Do I need to install or copy Kafka jar on Namenode? Actually, I don't want to install everything on Name node server. Please advise me. Thanks.
My Flink and Hadoop cluster info as below. Flink on NameNode Kafka,Zookeeper and FLink slave1 on Datanode1 Kafka,Zookeeper ,and Flink slave2 on Datanode2 Kafka, Zookeeper and Flink slave3 on Datanode3 On Thu, Jun 11, 2015 at 1:16 AM, Márton Balassi <balassi.mar...@gmail.com> wrote: > Dear Hawin, > > No problem, I am gald that you are giving our Kafka connector a try. :) > The dependencies listed look good. Can you run the example locally from > Eclipse with 'Run'? I suspect that maybe your Flink cluster does not have > the access to the kafka dependency then. > > As a quick test you could copy the kafka jars to the lib folder of your > Flink distribution on all the machines in your cluster. Everything that is > there goes to the classpath of Flink. Another workaround with be to build a > fat jar for your project containing all the dependencies with 'mvn > assembly:assembly'. Neither of these are beautiful but would help tracking > down the root cause. > > On Thu, Jun 11, 2015 at 10:04 AM, Hawin Jiang <hawin.ji...@gmail.com> > wrote: > >> Dear Marton >> >> >> >> Thanks for supporting again. >> >> I am running these examples at the same project and I am using Eclipse >> IDE to submit it to my Flink cluster. >> >> >> >> >> >> Here is my dependencies >> >> >> ****************************************************************************** >> >> <dependencies> >> >> <dependency> >> >> <groupId>*junit*</groupId> >> >> <artifactId>*junit*</artifactId> >> >> <version>4.12</version> >> >> <scope>test</scope> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>*flink*-java</artifactId> >> >> <version>0.9.0-milestone-1</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>*flink*-clients</artifactId> >> >> <version>0.9.0-milestone-1</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>*flink*-streaming-connectors</artifactId> >> >> <version>0.9.0-milestone-1</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.flink</groupId> >> >> <artifactId>*flink*-streaming-core</artifactId> >> >> <version>0.9.0-milestone-1</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.kafka</groupId> >> >> <artifactId>kafka_2.10</artifactId> >> >> <version>0.8.2.1</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.kafka</groupId> >> >> <artifactId>*kafka*-clients</artifactId> >> >> <version>0.8.2.1</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.hadoop</groupId> >> >> <artifactId>*hadoop*-*hdfs*</artifactId> >> >> <version>2.6.0</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.hadoop</groupId> >> >> <artifactId>*hadoop*-*auth*</artifactId> >> >> <version>2.6.0</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.hadoop</groupId> >> >> <artifactId>*hadoop*-common</artifactId> >> >> <version>2.6.0</version> >> >> </dependency> >> >> <dependency> >> >> <groupId>org.apache.hadoop</groupId> >> >> <artifactId>*hadoop*-core</artifactId> >> >> <version>1.2.1</version> >> >> </dependency> >> >> </dependencies> >> >> >> >> >> ***************************************************************************************** >> >> >> >> >> >> >> >> >> >> Best regards >> >> Email: hawin.ji...@gmail.com >> >> >> >> *From:* Márton Balassi [mailto:balassi.mar...@gmail.com] >> *Sent:* Thursday, June 11, 2015 12:58 AM >> *To:* user@flink.apache.org >> *Subject:* Re: Kafka0.8.2.1 + Flink0.9.0 issue >> >> >> >> Dear Hawin, >> >> >> >> This looks like a dependency issue, the java compiler does not find the >> kafka dependency. How are you trying to run this example? Is it from an IDE >> or submitting it to a flink cluster with bin/flink run? How do you define >> your dependencies, do you use maven or sbt for instance? >> >> >> >> Best, >> >> >> >> Marton >> >> >> >> On Thu, Jun 11, 2015 at 9:43 AM, Hawin Jiang <hawin.ji...@gmail.com> >> wrote: >> >> >> > >
