Turns out when I remove the explicit dependency on kafka_2.10 v. 0.8.1, then the dependencies are properly included. Guess there was a conflict somehow? I'll need to figure out if the rest of the code is fine with kafka_2.10 v. 0.8.2.0 as well.
On Mon, Apr 18, 2016 at 4:32 PM, Robert Schmidtke <ro.schmid...@gmail.com> wrote: > Hi Robert, > > thanks for your offer. After playing around a bit I would like to take it, > if you have the time: > https://github.com/robert-schmidtke/HiBench/blob/flink-streaming/src/streambench/flinkbench/pom.xml > > I would guess the POM is similar to the one in the sample project, yet > when building it, the jar does not contain all indirect dependencies. > > Thanks!! > > Robert > > On Mon, Apr 18, 2016 at 4:07 PM, Robert Metzger <rmetz...@apache.org> > wrote: > >> If you want you can share the pom of your project privately. >> >> On Mon, Apr 18, 2016 at 4:05 PM, Robert Schmidtke <ro.schmid...@gmail.com >> > wrote: >> >>> You're right, it does not. When including it the resulting jar has the >>> Kafka dependencies bundled. Now it's up to me to figure out the difference >>> between the sample project and the one I'm working on. >>> >>> Thanks! Really quick help. >>> >>> Robert >>> >>> On Mon, Apr 18, 2016 at 4:02 PM, Robert Metzger <rmetz...@apache.org> >>> wrote: >>> >>>> Hi, >>>> the problem with the posted project is that it doesn't have the Flink >>>> kafka connector as a dependency. >>>> >>>> On Mon, Apr 18, 2016 at 3:56 PM, Robert Schmidtke < >>>> ro.schmid...@gmail.com> wrote: >>>> >>>>> Hi Robert, >>>>> >>>>> thanks for your hints. I was not sure whether I was building a proper >>>>> fat jar, as I have not used the Flink Archetype for my project. However, I >>>>> have set up a sample project at >>>>> >>>>> https://github.com/robert-schmidtke/flink-test/ >>>>> >>>>> which is nothing more than the Quickstart Archetype plus the >>>>> instructions to bundle the Kafka dependencies. The resulting jars (mvn >>>>> clean package -Pbuild-jar and mvn clean package) do not contain the >>>>> org/apache/kafka/** classes. Can you have a quick look at the pom? >>>>> However, >>>>> as I said, it's verbatim Archetype+Flink Docs. >>>>> >>>>> Thanks a lot in advance! >>>>> >>>>> Robert >>>>> >>>>> >>>>> >>>>> On Mon, Apr 18, 2016 at 12:36 PM, Robert Metzger <rmetz...@apache.org> >>>>> wrote: >>>>> >>>>>> Hi, >>>>>> did you check your user jar if it contains the Kafka classes? >>>>>> Are you building a fat jar? Are you manually excluding any >>>>>> dependencies? >>>>>> >>>>>> Flink's 0.10.2 Kafka connector depends on Kafka 0.8.2.0 [1] which in >>>>>> turn depends on kafka-clients 0.8.2.0 [2]. And the "kafka-clients" >>>>>> dependency also contains the org.apache.kafka.common.Node class (the >>>>>> LegacyFetcher needs that class). >>>>>> >>>>>> >>>>>> [1] >>>>>> https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka/0.10.2/flink-connector-kafka-0.10.2.pom >>>>>> [2] >>>>>> https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom >>>>>> >>>>>> On Mon, Apr 18, 2016 at 7:42 AM, Balaji Rajagopalan < >>>>>> balaji.rajagopa...@olacabs.com> wrote: >>>>>> >>>>>>> I had fought with 0.8.0.2 kafka and flink 0.10.2 scala version 2.11, >>>>>>> was never able to get it working confounded with noclassdeffounderror, >>>>>>> moved to flink 1.0.0 with kafka 0.8.0.2 scala version 2.11 things >>>>>>> worked >>>>>>> for me, if moving to flink 1.0.0 is an option for you do so. >>>>>>> >>>>>>> balaji >>>>>>> >>>>>>> On Mon, Apr 18, 2016 at 3:19 AM, Robert Schmidtke < >>>>>>> ro.schmid...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi everyone, >>>>>>>> >>>>>>>> I have a Kafka cluster running on version 0.8.1, hence I'm using >>>>>>>> the FlinkKafkaConsumer081. When running my program, I saw a >>>>>>>> NoClassDefFoundError for org.apache.kafka.common.Node. So I packaged my >>>>>>>> binaries according to >>>>>>>> https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/cluster_execution.html#linking-with-modules-not-contained-in-the-binary-distribution, >>>>>>>> however I'm still seeing the error. >>>>>>>> >>>>>>>> I played around a bit and it turns out I have to package >>>>>>>> kafka-clients v. 0.8.2.0 instead of kafka_2.10 v. 0.8.1 with my >>>>>>>> program. Is >>>>>>>> there an error in the documentation or have I not figured out something >>>>>>>> properly? >>>>>>>> >>>>>>>> Thanks! >>>>>>>> Robert >>>>>>>> >>>>>>>> -- >>>>>>>> My GPG Key ID: 336E2680 >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> My GPG Key ID: 336E2680 >>>>> >>>> >>>> >>> >>> >>> -- >>> My GPG Key ID: 336E2680 >>> >> >> > > > -- > My GPG Key ID: 336E2680 > -- My GPG Key ID: 336E2680
