Hello Akash,
I will try by changing compression type and test if the issue persists, but
for now the issue is resolved I guess. I just changed replication_factor of
the topic to 3. It was like a 3-node kafka cluster and replication_factor
was configured 2 for the topic.
On Wed, Aug 28, 2024 at 2:55 PM Akash Jain <akashjain0...@gmail.com> wrote:
> Also what is the version of Java you are running?
>
> On Wed, Aug 28, 2024 at 11:44 AM Akash Jain <akashjain0...@gmail.com>
> wrote:
>
> > Hi Vikram, please share some more details:
> >
> > 1. producer version
> > 2. broker version
> > 3. broker side config you mentioned is "snappy", can you check what is
> > it on topic level as well
> > 4. do you face this issue when you use any other algorithm instead of
> > snappy, say lz4? Try with topic compression.type set to producer and
> > producer compression.type to anything other than snappy
> > 5. Do you face this issue when you disable compression?
> > 6. You also mention that you are facing this issue randomly, can you
> > elaborate more? Like you have different producer versions - and a
> > specific producer shows this problem? Or does a specific broker show
> this
> > problem? Have you observed any pattern at all?
> >
> > Do you see these exceptions in broker? I guess yes. Broker will
> decompress
> > the messages. It is likely because the version of snappy that is used in
> > producer is 'far away' from the version of snappy in broker.
> >
> > On Tue, Aug 27, 2024 at 8:00 AM Vikram Singh
> > <vikram.si...@nciportal.com.invalid> wrote:
> >
> >> Hello Akash,
> >>
> >> It's the same on the broker side as well.
> >>
> >> *Broker side config:-* compression.type=snappy
> >>
> >> On Mon, Aug 26, 2024 at 9:30 PM Akash Jain <akashjain0...@gmail.com>
> >> wrote:
> >>
> >> > And what is it on broker?
> >> >
> >> > On Sunday, August 25, 2024, Vikram Singh <vikram.si...@nciportal.com
> >> > .invalid>
> >> > wrote:
> >> >
> >> > > Hello Akash,
> >> > >
> >> > >
> >> > > Yes, i am using compression type in producer side configuration as
> >> > > mentioned below,
> >> > >
> >> > > And the compression type I am using is *snappy*.
> >> > >
> >> > >
> >> > > *Compression on producer :-*
> >> > >
> >> > > producerProperties.put("compression.type",CloudKafkaProducerConfig.
> >> > > COMPRESSION_TYPE);
> >> > >
> >> > > On Fri, Aug 23, 2024 at 5:56 PM Akash Jain <akashjain0...@gmail.com
> >
> >> > > wrote:
> >> > >
> >> > > > Hi Vikram. Can you share you code snippet? Are you using
> >> compression on
> >> > > > producer/broker side?
> >> > > >
> >> > > > On Friday, August 23, 2024, Vikram Singh <
> >> vikram.si...@nciportal.com
> >> > > > .invalid>
> >> > > > wrote:
> >> > > >
> >> > > > > Hello,
> >> > > > >
> >> > > > > I am facing issues while producing messages on kafka topics. I
> am
> >> > > facing
> >> > > > > this issue randomly. Please help me by referring to the logs
> >> below.
> >> > > > >
> >> > > > > Logs :-
> >> > > > > 1). ERROR [ReplicaManager broker=0] Error processing append
> >> operation
> >> > > on
> >> > > > > partition MHM_CLZ_COM-AWS_123-7 (kafka.server.ReplicaManager)
> >> > > > >
> >> > > > > org.apache.kafka.common.KafkaException: Failed to decompress
> >> record
> >> > > > stream
> >> > > > >
> >> > > > > 2). Caused by: java.io.IOException: FAILED_TO_UNCOMPRESS(5)
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>
