ByteArrayDeserializer is not an instance of org.apache.kafka.common.serialization.Deserializer

Tue, 17 Nov 2020 00:47:46 -0800 

flink 1.11.1-2.12消费kafka0.10.1.1的时候报错。
在idea调试没报错，提交到yarn集群就报错了。求助。


使用的consumer如下：
val logConsumer: FlinkKafkaConsumer010[String] = new 
FlinkKafkaConsumer010[String](topic, new SimpleStringSchema(), properties)


pom文件导入的依赖如下：
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-connector-kafka-0.10_2.12</artifactId>
            <version>1.11.1</version>
        </dependency>



报错如下：

2020-11-17 16:39:37

org.apache.kafka.common.KafkaException: Failed to construct kafka consumer

at 
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:718)

at 
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:597)

at 
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:579)

at 
org.apache.flink.streaming.connectors.kafka.internal.Kafka010PartitionDiscoverer.initializeConnections(Kafka010PartitionDiscoverer.java:58)

at 
org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)

at 
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)

at 
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)

at 
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)

at 
org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)

at 
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:473)

at 
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)

at 
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:469)

at 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:522)

at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)

at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)

at java.lang.Thread.run(Thread.java:745)

Caused by: org.apache.kafka.common.KafkaException: 
org.apache.flink.kafka010.shaded.org.apache.kafka.common.serialization.ByteArrayDeserializer
 is not an instance of org.apache.kafka.common.serialization.Deserializer

at 
org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:205)

at 
org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:637)

... 15 more







--

kingdomad

回复