Prashant created KAFKA-9542:
-------------------------------
Summary: ZSTD Compression Not Working
Key: KAFKA-9542
URL: https://issues.apache.org/jira/browse/KAFKA-9542
Project: Kafka
Issue Type: Bug
Components: compression
Affects Versions: 2.3.0
Environment: Linux, CentOS
Reporter: Prashant
I enabled zstd compression at producer by adding "compression.type=zstd" in
producer config. When try to run it, producer fails with
"org.apache.kafka.common.errors.UnknownServerException: The server experienced
an unexpected error when processing the request"
In Broker Logs, I could find following exception:
[2020-02-12 11:48:04,623] ERROR [ReplicaManager broker=1] Error processing
append operation on partition load_logPlPts-6 (kafka.server.ReplicaManager)
org.apache.kafka.common.KafkaException: java.lang.NoClassDefFoundError: Could
not initialize class
org.apache.kafka.common.record.CompressionType$ZstdConstructors
at
org.apache.kafka.common.record.CompressionType$5.wrapForInput(CompressionType.java:133)
at
org.apache.kafka.common.record.DefaultRecordBatch.compressedIterator(DefaultRecordBatch.java:257)
at
org.apache.kafka.common.record.DefaultRecordBatch.iterator(DefaultRecordBatch.java:324)
at
scala.collection.convert.Wrappers$JIterableWrapper.iterator(Wrappers.scala:54)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at
kafka.log.LogValidator$$anonfun$validateMessagesAndAssignOffsetsCompressed$1.apply(LogValidator.scala:269)
at
kafka.log.LogValidator$$anonfun$validateMessagesAndAssignOffsetsCompressed$1.apply(LogValidator.scala:261)
at scala.collection.Iterator$class.foreach(Iterator.scala:891)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
at
kafka.log.LogValidator$.validateMessagesAndAssignOffsetsCompressed(LogValidator.scala:261)
at
kafka.log.LogValidator$.validateMessagesAndAssignOffsets(LogValidator.scala:72)
at kafka.log.Log$$anonfun$append$2.liftedTree1$1(Log.scala:869)
at kafka.log.Log$$anonfun$append$2.apply(Log.scala:868)
at kafka.log.Log$$anonfun$append$2.apply(Log.scala:850)
at kafka.log.Log.maybeHandleIOException(Log.scala:2065)
at kafka.log.Log.append(Log.scala:850)
at kafka.log.Log.appendAsLeader(Log.scala:819)
at kafka.cluster.Partition$$anonfun$14.apply(Partition.scala:771)
at kafka.cluster.Partition$$anonfun$14.apply(Partition.scala:759)
at kafka.utils.CoreUtils$.inLock(CoreUtils.scala:253)
This is fresh broker installed on "CentOS Linux" v7. This doesn't seem to be a
classpath issue as same package is working on MacOS.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
