Hi Smile, you missed installing the pom provided by mvnrepository.org [1]. Maven will install a basic pom if none is provided [2]. This basic pom file will not include any dependencies. You should be able to fix your problem by running your command above but adding the -DpomFile property with the pom file provided in [1]:
mvn install:install-file -DgroupId=io.confluent -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar -Dfile=kafka-avro-serializer-5.5.2.jar -DpomFile=kafka-avro-serializer-5.5.2.pom [1] https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom [2] https://maven.apache.org/plugins/maven-install-plugin/install-file-mojo.html#pomFile On Mon, Jan 25, 2021 at 8:25 AM Smile@LETTers <letters_sm...@163.com> wrote: > Hi Matthias, > Sorry for my miss leading. I mean kafka-schema-serializer rather than > kafka-avro-serializer. > > io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe is in > kafka-schema-serializer and kafka-schema-serializer should be a dependency > of kafka-avro-serializer according to their pom.xml files(see [1], [2]). > I couldn't resolve a valid kafka-avro-serializer.jar in my mirror so I > downloaded it manually from [3] and installed it using: > mvn install:install-file -DgroupId=io.confluent > -DartifactId=kafka-avro-serializer -Dversion=5.5.2 -Dpackaging=jar > -Dfile=kafka-avro-serializer-5.5.2.jar > After that, I tried to build Flink and got the above exceptions. Then I > tried to add the dependency of kafka-schema-serializer to > flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml (also > manually installed it to my local maven repo) and everything went well. > I also tried to remove it from the pom.xml after installing, and the > exception came back. > Maybe there was something wrong with the manually-installed > kafka-avro-serializer? > > > [1]. > https://mvnrepository.com/artifact/io.confluent/kafka-schema-serializer/usages > [2]. > https://packages.confluent.io/maven/io/confluent/kafka-avro-serializer/5.5.2/kafka-avro-serializer-5.5.2.pom > [3]. https://mvnrepository.com/artifact/io.confluent/kafka-avro-serializer > > At 2021-01-22 21:22:51, "Matthias Pohl" <matth...@ververica.com> wrote: > > Hi Smile, > Have you used a clean checkout? I second Robert's statement considering > that the dependency you're talking about is already part > of flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/pom.xml. It > also has the correct scope set both in master and release-1.12. > > Best, > Matthias > > On Fri, Jan 22, 2021 at 10:04 AM Smile@LETTers <letters_sm...@163.com> > wrote: > >> Yes, I've tried from both the root directory and the sub module. Neither >> or them works. And the error messages are the same. >> >> At 2021-01-21 23:22:12, "Robert Metzger" <rmetz...@apache.org> wrote: >> >> Since our CI system is able to build Flink, I believe it's a local issue. >> >> Are you sure that the build is failing when you build Flink from the root >> directory (not calling maven from within a maven module?) >> >> On Tue, Jan 19, 2021 at 11:19 AM Smile@LETTers <letters_sm...@163.com> >> wrote: >> >>> Hi, >>> I got an error when tried to compile & package Flink (version 1.12 & >>> current master). >>> It can be reproduced by run 'mvn clean test' under >>> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka. >>> >>> It seems that a necessary dependency for test scope was missing and some >>> classes can not be found. >>> After adding the dependency kafka-avro-serializer to the pom of >>> flink-end-to-end-tests/flink-end-to-end-tests-common-kafka everything goes >>> well. >>> >>> And I just wonder that is this a bug or I missed some local setting? >>> >>> Best regards. >>> Smile >>> >>> ------------------------------ >>> Error logs attached: >>> >>> >>> [INFO] --------< org.apache.flink:flink-end-to-end-tests-common-kafka >>> >-------- >>> [INFO] Building Flink : E2E Tests : Common Kafka 1.13-SNAPSHOT >>> [INFO] --------------------------------[ jar >>> ]--------------------------------- >>> Downloading ... >>> [INFO] >>> [INFO] --- maven-clean-plugin:3.1.0:clean (default-clean) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] Deleting >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target >>> [INFO] >>> [INFO] --- maven-checkstyle-plugin:2.17:check (validate) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- spotless-maven-plugin:2.4.2:check (spotless-check) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce >>> (enforce-maven-version) @ flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-maven) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-snakeyaml) >>> @ flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (ban-unsafe-jackson) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (forbid-log4j-1) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- maven-enforcer-plugin:3.0.0-M1:enforce (enforce-versions) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- gmavenplus-plugin:1.8.1:execute (merge-categories) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] Using plugin classloader, includes GMavenPlus classpath. >>> [INFO] Using Groovy 2.5.12 to perform execute. >>> includes: org.apache.flink.tests.util.categories.Dummy >>> excludes: >>> [INFO] >>> [INFO] --- directory-maven-plugin:0.1:highest-basedir (directories) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] Highest basedir set to: >>> /Users/smile/Downloads/W/code/flink/apache/master/flink >>> [INFO] >>> [INFO] --- maven-remote-resources-plugin:1.5:process >>> (process-resource-bundles) @ flink-end-to-end-tests-common-kafka --- >>> [INFO] >>> [INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] Using 'UTF-8' encoding to copy filtered resources. >>> [INFO] skip non existing resourceDirectory >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/resources >>> [INFO] Copying 3 resources >>> [INFO] >>> [INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ >>> flink-end-to-end-tests-common-kafka --- >>> [INFO] Compiling 5 source files to >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/classes >>> [INFO] >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java >>> uses unchecked or unsafe operations. >>> [INFO] >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/main/java/org/apache/flink/tests/util/kafka/KafkaContainerClient.java: >>> Recompile with -Xlint:unchecked for details. >>> [INFO] >>> [INFO] --- maven-resources-plugin:3.1.0:testResources >>> (default-testResources) @ flink-end-to-end-tests-common-kafka --- >>> [INFO] Using 'UTF-8' encoding to copy filtered resources. >>> [INFO] Copying 2 resources >>> [INFO] Copying 3 resources >>> [INFO] >>> [INFO] --- maven-compiler-plugin:3.8.0:testCompile (default-testCompile) >>> @ flink-end-to-end-tests-common-kafka --- >>> [INFO] Compiling 4 source files to >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/target/test-classes >>> [INFO] ------------------------------------------------------------- >>> [ERROR] COMPILATION ERROR : >>> [INFO] ------------------------------------------------------------- >>> [ERROR] >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] >>> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe >>> class file for io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe >>> not found >>> [INFO] 1 error >>> [INFO] ------------------------------------------------------------- >>> [INFO] >>> ------------------------------------------------------------------------ >>> [INFO] BUILD FAILURE >>> [INFO] >>> ------------------------------------------------------------------------ >>> [INFO] Total time: 29.842 s >>> [INFO] Finished at: 2021-01-19T18:04:39+08:00 >>> [INFO] >>> ------------------------------------------------------------------------ >>> [ERROR] Failed to execute goal >>> org.apache.maven.plugins:maven-compiler-plugin:3.8.0:testCompile >>> (default-testCompile) on project flink-end-to-end-tests-common-kafka: >>> Compilation failure >>> [ERROR] >>> /Users/smile/Downloads/W/code/flink/apache/master/flink/flink-end-to-end-tests/flink-end-to-end-tests-common-kafka/src/test/java/org/apache/flink/tests/util/kafka/SQLClientSchemaRegistryITCase.java:[113,20] >>> cannot access io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe >>> [ERROR] class file for >>> io.confluent.kafka.serializers.AbstractKafkaSchemaSerDe not found >>> [ERROR] >>> [ERROR] -> [Help 1] >>> [ERROR] >>> [ERROR] To see the full stack trace of the errors, re-run Maven with the >>> -e switch. >>> [ERROR] Re-run Maven using the -X switch to enable full debug logging. >>> [ERROR] >>> [ERROR] For more information about the errors and possible solutions, >>> please read the following articles: >>> [ERROR] [Help 1] >>> http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException >>> >> >
