Dariusz Seweryn created NIFI-14415:
--------------------------------------
Summary: ConsumeKinesisStream skips aggregated records
Key: NIFI-14415
URL: https://issues.apache.org/jira/browse/NIFI-14415
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 2.3.0
Reporter: Dariusz Seweryn
Assignee: Dariusz Seweryn
On 2025-02-11, aws-kinesis-client-library-version was bumped from 2.6.0 to
2.6.1 which transitively updated protobuf-java from 3.21.12 to 4.27.0. Along
this bump protobuf-java dependency override was not removed, pinning to version
3.25.6.
As a result aggregated records are skipped with following warning:
{quote}runtime-server-1 | [APP] 2025-04-01 14:06:05,990 WARN
[ConsumeKinesisStream-Scheduler-runtime-server:bb1bafd3-cde7-4be6-a3a1-dc0a698baa10]
software.amazon.kinesis.lifecycle.ShardConsumer single_stream_mode : Exception
occurred while dispatching incoming data. The incoming data has been skipped
runtime-server-1 | java.lang.NoClassDefFoundError:
com/google/protobuf/RuntimeVersion$RuntimeDomain
runtime-server-1 | at
software.amazon.kinesis.retrieval.kpl.Messages$AggregatedRecord.<clinit>(Messages.java:1909)
runtime-server-1 | at
software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:126)
runtime-server-1 | at
software.amazon.kinesis.retrieval.AggregatorUtil.deaggregate(AggregatorUtil.java:70)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ProcessTask.deaggregateAnyKplRecords(ProcessTask.java:204)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ProcessTask.call(ProcessTask.java:158)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.executeTask(ShardConsumer.java:412)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.processData(ShardConsumer.java:396)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumer.handleInput(ShardConsumer.java:201)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:181)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.ShardConsumerSubscriber.onNext(ShardConsumerSubscriber.java:36)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:56)
runtime-server-1 | at
software.amazon.kinesis.lifecycle.NotifyingSubscriber.onNext(NotifyingSubscriber.java:27)
runtime-server-1 | at
io.reactivex.rxjava3.internal.util.HalfSerializer.onNext(HalfSerializer.java:46)
runtime-server-1 | at
io.reactivex.rxjava3.internal.subscribers.StrictSubscriber.onNext(StrictSubscriber.java:97)
runtime-server-1 | at
io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$ObserveOnSubscriber.runAsync(FlowableObserveOn.java:403)
runtime-server-1 | at
io.reactivex.rxjava3.internal.operators.flowable.FlowableObserveOn$BaseObserveOnSubscriber.run(FlowableObserveOn.java:178)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker$BooleanRunnable.run(ExecutorScheduler.java:324)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.runEager(ExecutorScheduler.java:289)
runtime-server-1 | at
io.reactivex.rxjava3.internal.schedulers.ExecutorScheduler$ExecutorWorker.run(ExecutorScheduler.java:250)
runtime-server-1 | at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
runtime-server-1 | at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
runtime-server-1 | at java.base/java.lang.Thread.run(Thread.java:1583)
runtime-server-1 | Caused by: java.lang.ClassNotFoundException:
com.google.protobuf.RuntimeVersion$RuntimeDomain
runtime-server-1 | at
java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445)
runtime-server-1 | at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:593)
runtime-server-1 | at
java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:526)
runtime-server-1 | ... 22 common frames omitted
{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
