[
https://issues.apache.org/jira/browse/KAFKA-15947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ludo resolved KAFKA-15947.
--------------------------
Fix Version/s: 3.6.1
Resolution: Duplicate
> Null pointer on LZ4 compression since Kafka 3.6
> -----------------------------------------------
>
> Key: KAFKA-15947
> URL: https://issues.apache.org/jira/browse/KAFKA-15947
> Project: Kafka
> Issue Type: Bug
> Components: compression
> Affects Versions: 3.6.0
> Reporter: Ludo
> Priority: Major
> Fix For: 3.6.1
>
>
> I have a Kafka Stream application running well since month using client
> version {{3.5.1 }}with 3.5.1 (bitnami image: {{bitnami/3.5.1-debian-11-r44)}}
> using{{ compression.type: "lz4"}}
> I've recently updated a my kafka server to kafka 3.6 (bitnami image:
> {{{}bitnami/kafka:3.6.0-debian-11-r0){}}}.
>
> The startup is working well for days, and after some time, Kafka Stream crash
> and Kafka output a lot of NullPointerException on the console:
>
> {code:java}
> org.apache.kafka.common.KafkaException: java.lang.NullPointerException:
> Cannot invoke "java.nio.ByteBuffer.hasArray()" because
> "this.intermediateBufRef" is null
> at
> org.apache.kafka.common.record.CompressionType$4.wrapForInput(CompressionType.java:134)
> at
> org.apache.kafka.common.record.DefaultRecordBatch.recordInputStream(DefaultRecordBatch.java:273)
> at
> org.apache.kafka.common.record.DefaultRecordBatch.compressedIterator(DefaultRecordBatch.java:277)
> at
> org.apache.kafka.common.record.DefaultRecordBatch.skipKeyValueIterator(DefaultRecordBatch.java:352)
> at
> org.apache.kafka.storage.internals.log.LogValidator.validateMessagesAndAssignOffsetsCompressed(LogValidator.java:358)
> at
> org.apache.kafka.storage.internals.log.LogValidator.validateMessagesAndAssignOffsets(LogValidator.java:165)
> at kafka.log.UnifiedLog.$anonfun$append$2(UnifiedLog.scala:805)
> at kafka.log.UnifiedLog.append(UnifiedLog.scala:1845)
> at kafka.log.UnifiedLog.appendAsLeader(UnifiedLog.scala:719)
> at
> kafka.cluster.Partition.$anonfun$appendRecordsToLeader$1(Partition.scala:1313)
> at kafka.cluster.Partition.appendRecordsToLeader(Partition.scala:1301)
> at
> kafka.server.ReplicaManager.$anonfun$appendToLocalLog$6(ReplicaManager.scala:1210)
> at
> scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
> at
> scala.collection.mutable.HashMap.$anonfun$foreach$1(HashMap.scala:149)
> at scala.collection.mutable.HashTable.foreachEntry(HashTable.scala:237)
> at scala.collection.mutable.HashTable.foreachEntry$(HashTable.scala:230)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:44)
> at scala.collection.mutable.HashMap.foreach(HashMap.scala:149)
> at scala.collection.TraversableLike.map(TraversableLike.scala:286)
> at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
> at scala.collection.AbstractTraversable.map(Traversable.scala:108)
> at
> kafka.server.ReplicaManager.appendToLocalLog(ReplicaManager.scala:1198)
> at
> kafka.server.ReplicaManager.$anonfun$appendRecords$18$adapted(ReplicaManager.scala:754)
> at
> kafka.server.KafkaRequestHandler$.$anonfun$wrap$3(KafkaRequestHandler.scala:73)
> at kafka.server.KafkaRequestHandler.run(KafkaRequestHandler.scala:130)
> at java.base/java.lang.Thread.run(Thread.java:833)
> Caused by: java.lang.NullPointerException: Cannot invoke
> "java.nio.ByteBuffer.hasArray()" because "this.intermediateBufRef" is null
> at
> org.apache.kafka.common.utils.ChunkedBytesStream.<init>(ChunkedBytesStream.java:89)
> at
> org.apache.kafka.common.record.CompressionType$4.wrapForInput(CompressionType.java:132)
> ... 25 more {code}
> At the same time the Kafka Stream raise this error:
>
> {code:java}
> org.apache.kafka.streams.errors.StreamsException: Error encountered sending
> record to topic kestra_workertaskresult for task 3_6 due
> to:org.apache.kafka.common.errors.UnknownServerException: The server
> experienced an unexpected error when processing the request.Written offsets
> would not be recorded and no more records would be sent since this is a fatal
> error.at
> org.apache.kafka.streams.processor.internals.RecordCollectorImpl.recordSendError(RecordCollectorImpl.java:297)at
>
> org.apache.kafka.streams.processor.internals.RecordCollectorImpl.lambda$send$1(RecordCollectorImpl.java:284)at
>
> org.apache.kafka.clients.producer.KafkaProducer$AppendCallbacks.onCompletion(KafkaProducer.java:1505)at
>
> org.apache.kafka.clients.producer.internals.ProducerBatch.completeFutureAndFireCallbacks(ProducerBatch.java:273)at
>
> org.apache.kafka.clients.producer.internals.ProducerBatch.done(ProducerBatch.java:234)at
>
> org.apache.kafka.clients.producer.internals.ProducerBatch.completeExceptionally(ProducerBatch.java:198)at
>
> org.apache.kafka.clients.producer.internals.Sender.failBatch(Sender.java:772)at
>
> org.apache.kafka.clients.producer.internals.Sender.failBatch(Sender.java:757)at
>
> org.apache.kafka.clients.producer.internals.Sender.failBatch(Sender.java:709)at
>
> org.apache.kafka.clients.producer.internals.Sender.completeBatch(Sender.java:648)at
>
> org.apache.kafka.clients.producer.internals.Sender.lambda$null$1(Sender.java:589)at
> java.base/java.util.ArrayList.forEach(Unknown Source)at
> org.apache.kafka.clients.producer.internals.Sender.lambda$handleProduceResponse$2(Sender.java:576)at
> java.base/java.lang.Iterable.forEach(Unknown Source)at
> org.apache.kafka.clients.producer.internals.Sender.handleProduceResponse(Sender.java:576)at
>
> org.apache.kafka.clients.producer.internals.Sender.lambda$sendProduceRequest$5(Sender.java:850)at
>
> org.apache.kafka.clients.ClientResponse.onComplete(ClientResponse.java:154)at
> org.apache.kafka.clients.NetworkClient.completeResponses(NetworkClient.java:594)at
> org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:586)at
> org.apache.kafka.clients.producer.internals.Sender.runOnce(Sender.java:328)at
> org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:243)at
> java.base/java.lang.Thread.run(Unknown Source)Caused by:
> org.apache.kafka.common.errors.UnknownServerException: The server experienced
> an unexpected error when processing the request.
> {code}
> The error will loop on the Kafka Stream (pod are restarted, I supposed that
> topic are consumed on the same offset, and crash with the same error on both
> server and client).
> Always, when I restart Kafka server, every goes well and I don't have any
> more issues.
> The error is mostly transitive and happen every few days without any others
> solution than restart the server.
>
>
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
