Hi all! I accidently posted this on the dev mailing list yesterday, when it's much better off here. Can anybody help me?
When I run a pig script I get: java.lang.InstantiationError: org.apache.avro.io.BinaryEncoder at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462) at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522) Caused by: java.lang.InstantiationError: org.apache.avro.io.BinaryEncoder at kafka.bridge.pig.AvroKafkaStorage.prepareToWrite(AvroKafkaStorage.java:77) I am using a HDP 2.2 installation which comes with Kafka 0.8.1. I try to push a single string from Pig into Kafka and follow these instructions: https://github.com/apache/kafka/blob/43b92f8b1ce8140c432edf11b0c842f5fbe04120/contrib/hadoop-producer/README.md The script looks like: REGISTER /usr/hdp/current/kafka-broker/contrib/lib/kafka-hadoop-producer-0.8.1.2.2.0.0-2041.jar; REGISTER /usr/hdp/current/pig-client/lib/avro-1.4.1.jar; REGISTER /usr/hdp/current/pig-client/piggybank.jar; REGISTER /usr/hdp/current/kafka-broker/libs/kafka_2.10-0.8.1.2.2.0.0-2041.jar; REGISTER /usr/hdp/current/pig-client/lib/jackson-core-asl-1.8.8.jar; REGISTER /usr/hdp/current/pig-client/lib/jackson-mapper-asl-1.8.8.jar; REGISTER /usr/hdp/current/kafka-broker/libs/scala-library-2.10.4.jar; STORE my_data INTO 'kafka://MY-BROKER-HOST-NAME:6667/myTopic' USING kafka.bridge.pig.AvroKafkaStorage('"string"'); MY-BROKER-HOST-NAME is one of my Kafka brokers, myTopic is an existing topic. The version 1.4.1 of avro is not deployed by default, but I deployed it and registered it instead of avro 1.7.5, because I first thought this will solve my problem, but the same problem occurs. Also, I noticed, that the same problom still occurs, when I do not register avro at all. When I look into the avro documentation I can see, that the BinaryEncoder class is abstract, but nevertheless is tried to be instantiated in the AvroKakfaStorage class. So the error is not a surprise. Is there anything I am missing to make this work? Am I doing something wrong? Best regards, Stefan
