I went through the logs segment of the topic and i see that the data from
the producerID still exists in the topic and the new messages received are
well with in the retention period ( 336 hr).
Starting offset: 359
offset: 359 position: 0 CreateTime: 1541667838856 isvalid: true keysize: 39
valuesize: 500 magic: 2 compresscodec: GZIP producerId: 39002
producerEpoch: 0 sequence: 0 isTransactional: false headerKeys: [] key:
${currentDate[yyyy-mm-dd hh:MM:ss.SSS]} payload: {"table":
offset: 360 position: 405 CreateTime: 1542334682507 isvalid: true keysize:
39 valuesize: 501 magic: 2 compresscodec: GZIP producerId: 41003
producerEpoch: 0 sequence: 0 isTransactional: false headerKeys: [] key:
${currentDate[yyyy-mm-dd hh:MM:ss.SSS]} payload: {"table":
offset: 361 position: 405 CreateTime: 1542334682507 isvalid: true keysize:
39 valuesize: 500 magic: 2 compresscodec: GZIP producerId: 41003
producerEpoch: 0 sequence: 1 isTransactional: false headerKeys: [] key:
${currentDate[yyyy-mm-dd hh:MM:ss.SSS]} payload: {"table":"
I would be grateful if someone can help me find the root cause
On Fri, Nov 16, 2018 at 11:47 AM Faisal Durrani <te04.0...@gmail.com> wrote:
> Hi Users,
>
> Apologies if this question has been asked before but i have not been able
> to find the answer.
>
> I am seeing this error in the Kafka logs :
>
> [ReplicaManager broker=1001] Error processing append operation on
> partition TABLE-0 (kafka.server.ReplicaManager)
> org.apache.kafka.common.errors.UnknownProducerIdException: Found no record
> of producerId=39002 on the broker. It is possible that the last message
> with the producerId=39002 has been removed due to hitting the retention
> limit.
>
> This is the 2nd time we are seeing this error , we changed the retention
> limit to a larger value when we got the error the first time but its back
> now. We have 5 node Kafka cluster with Oracle golden gate as producer. The
> topic TABLE as 5 partition but at this point the data is not getting
> distributed on the cluster because( because Oracle is sending the same
> kafka.key for all the messages) . The compression type is set to 'producer'
> and at GG its set to gzip.
>
> Can you let me know what can we do to avoid getting this error as it is
> causing the GG to go Abend.
>
> Regards,
> Faisal
>
>
