Hi Dongjin,
Thank you for your very informative reply! Here's the answer to your questions: - What is the platform you are using? - We're currently using the following: Linux 4.4.0-89-generic #112-Ubuntu - Do you have enough space or permission for the temp directory? - Yes our application has permission and we have an approximate available space of 7.7gb inside the temp directory. - The error occurs repeatedly? - This error has only occurred once so far and we were not able to replicate this issue. Best regards, Loren Sion On Thu, Mar 18, 2021 at 12:19 PM Dongjin Lee <dong...@apache.org> wrote: > Hi Loren, > > > This error occurs when your application fails to load the libzstd-jni > library file from the temporary library. > > > > - zstd-jni v1.3.5-4 (included in Kafka 2.1.0): > > https://github.com/luben/zstd-jni/blob/v1.3.5-4/src/main/java/com/github/luben/zstd/util/Native.java#L101 > - zstd-jni lastest: > > https://github.com/luben/zstd-jni/blob/master/src/main/java/com/github/luben/zstd/util/Native.java#L137 > > > In short, zstd-jni works like the following: > > > > 1. It includes all supported platform's shared library in its jar. > 2. When initializing, it copies the appropriate library into the temp > directory. > 3. Load the extracted library into the memory. > > > So, It would be good to check: > > > > 1. What is the platform you are using? > 2. Do you have enough space or permission for the temp directory? > 3. The error occurs repeatedly? > > > Please have a check and give me a reply. (disclaimer: I added zstd support > to Apache Kafka.) > > > Thanks, > > Dongjin > > On Thu, Mar 18, 2021 at 12:17 AM Loren Abigail Sion <sion.lo...@gmail.com> > wrote: > > > Good day, > > > > > > We're currently in the process of implementing our application with kafka > > compression type ZStandard (zstd). > > > > However during the testing process the consumer encountered this error: > > > > [ERROR] (consumer-1) > > > > > org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer > > - Container exception > > org.apache.kafka.common.KafkaException: Received exception when fetching > > the next record from dp.-------------------. If needed, please seek past > > the record to continue consumption. > > at > > > > > org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.fetchRecords(Fetcher.java:1228) > > at > > > > > org.apache.kafka.clients.consumer.internals.Fetcher$PartitionRecords.access$1400(Fetcher.java:1096) > > at > > > > > org.apache.kafka.clients.consumer.internals.Fetcher.fetchRecords(Fetcher.java:544) > > at > > > > > org.apache.kafka.clients.consumer.internals.Fetcher.fetchedRecords(Fetcher.java:505) > > at > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.pollForFetches(KafkaConsumer.java:1225) > > at > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1188) > > at > > > > > org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1123) > > at > > > > > org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:532) > > at > > > > > java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) > > at > > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > at java.base/java.lang.Thread.run(Thread.java:834) > > Caused by: org.apache.kafka.common.KafkaException: > > java.lang.ExceptionInInitializerError: Cannot unpack libzstd-jni > > > > *Here's the version for the producer and consumer:* > > > > Producer Kafka Client Version (using ZStandard compression): 2.5.1 > > Consumer Kafka Client Version: 2.1.0 > > > > Could you help us identify what caused this error? Do we need to upgrade > > the version on the consumer side? > > > > > > Best Regards, > > > > Loren Sion > > > > > -- > *Dongjin Lee* > > *A hitchhiker in the mathematical world.* > > > > *github: <http://goog_969573159/>github.com/dongjinleekr > <https://github.com/dongjinleekr>keybase: https://keybase.io/dongjinleekr > <https://keybase.io/dongjinleekr>linkedin: kr.linkedin.com/in/dongjinleekr > <https://kr.linkedin.com/in/dongjinleekr>speakerdeck: > speakerdeck.com/dongjin > <https://speakerdeck.com/dongjin>* >