Hi Harper,

For the producers, I am using 1.1.0. As for the brokers, I do not have idea
as we are using Microsoft Event Hubs as our broker. Here's the error logs
that I have when I run my producer and consumers:

19/07/12 15:22:20.354 ERROR
reactor.kafka.receiver.internals.DefaultKafkaReceiver: Unexpected exception
org.apache.kafka.common.protocol.types.SchemaException: Error reading field
'topics': Error reading array of size 825844834, only 33 bytes available
at org.apache.kafka.common.protocol.types.Schema.read(Schema.java:76)
at
org.apache.kafka.clients.consumer.internals.ConsumerProtocol.deserializeSubscription(ConsumerProtocol.java:96)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.performAssignment(ConsumerCoordinator.java:340)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.onJoinLeader(AbstractCoordinator.java:520)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.access$1100(AbstractCoordinator.java:93)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$JoinGroupResponseHandler.handle(AbstractCoordinator.java:472)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$JoinGroupResponseHandler.handle(AbstractCoordinator.java:455)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:822)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:802)
at
org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:204)
at
org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:167)
at
org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:127)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:563)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:390)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:293)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233)
at
org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:193)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:364)
at
org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:316)
at
org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:290)
at
org.apache.kafka.clients.consumer.KafkaConsumer.pollOnce(KafkaConsumer.java:1149)
at
org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1115)
at
reactor.kafka.receiver.internals.DefaultKafkaReceiver$PollEvent.run(DefaultKafkaReceiver.java:446)
at
reactor.kafka.receiver.internals.DefaultKafkaReceiver.doEvent(DefaultKafkaReceiver.java:377)
at
reactor.kafka.receiver.internals.DefaultKafkaReceiver.lambda$start$16(DefaultKafkaReceiver.java:319)
at reactor.core.publisher.LambdaSubscriber.onNext(LambdaSubscriber.java:130)
at
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.runAsync(FluxPublishOn.java:396)
at
reactor.core.publisher.FluxPublishOn$PublishOnSubscriber.run(FluxPublishOn.java:480)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:84)
at reactor.core.scheduler.WorkerTask.call(WorkerTask.java:37)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Reading a bit deeper into it, I think it could be a problem for the
consumers, not the producers.

On Tue, Jul 23, 2019 at 1:41 PM Harper Henn <harper.h...@datto.com> wrote:

> Hi Kyla,
>
> What message format version are your producers and brokers using?
>
> That exception looks very similar to:
>
> https://stackoverflow.com/questions/39227213/why-kafka-0-10-console-producer-cannot-send-messages-to-kafka-0-9
>
> On Mon, Jul 22, 2019 at 6:10 AM Kyla Palos <kyla.pa...@gmail.com> wrote:
>
> > I have a producer that would send messages to a topic with 4 partitions.
> > Then, 4 consumers would consume from that topic. Most of the time, it
> works
> > properly. But then, I would sometimes experience this error:
> > org.apache.kafka.common.protocol.types.SchemaException: Error reading
> field
> > 'topics': Error reading array of size 825844834, only 33 bytes available
> >
> > Can anyone give me a background on why this is happening and how to fix
> > this?
> >
> > Thanks!
> >
>

Reply via email to