[
https://issues.apache.org/jira/browse/KAFKA-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772847#comment-17772847
]
Greg Harris commented on KAFKA-15564:
-------------------------------------
Hi [~hemanthsavasere] the offsets translation behavior was changed in
KAFKA-12468 to avoid a data loss bug. You will need to lower the
`offset.lag.max` if you want more precise consumer offset translation at the
end of the topic.
> Kafka 3.5.1- Mirror Maker 2 replicating the wrong consumer group offsets in
> destination
> ----------------------------------------------------------------------------------------
>
> Key: KAFKA-15564
> URL: https://issues.apache.org/jira/browse/KAFKA-15564
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
> Affects Versions: 3.5.1
> Reporter: Hemanth Savasere
> Priority: Major
> Attachments: Destination Consumer Group Offsets Describe.txt, MM2
> 3.5.1 Logs.txt, Source Consumer Group Offsets Describe.txt,
> docker-compose.yml, mm2.properties
>
>
> Issue : Mirror Maker 2 not replicating the proper consumer group offsets when
> replication happens from source to destination kafka brokers
> Steps to Reproduce :
> # Created 2 Kafka clusters locally using the attached docker-compose.yml
> file. Was using the confluent platform 7.2.1 docker images
> # Had added the below section in docker file to create 20 topics and produce
> randomly between 1000 to 2000 messages in each topic, and then consume the
> same messages using consumer-groups.
> {code:java}
> /etc/confluent/docker/run &
> sleep 20
> for i in {1..20}; do
> kafka-topics --create --bootstrap-server kafka-1:9092
> --replication-factor 1 --partitions 1 --topic topic$$i
> done
> for i in {1..20}; do
> num_msgs=$$(shuf -i 1000-2000 -n 1)
> seq 1 $$num_msgs | kafka-console-producer --broker-list
> kafka-1:9092 --topic topic$$i
> done
> for i in {1..20}; do
> timeout 10 kafka-console-consumer --bootstrap-server kafka-1:9092
> --topic topic$$i --group consumer-group$$i
> done
> wait
> {code}
> # Ran the Mirror Maker 2 using the connect-mirror-maker.sh script after
> downloading the Kafka 3.5.1 release binaries. Verified the 3.5.1 version was
> running and commitID 2c6fb6c54472e90a was shown in attached MM2 logs file.
> NOTE : Have attached the Kafka Docker file, the mirror maker 2 logs and
> mirror maker 2 properties file. Also, have attached the describe command
> output of all the consumer groups present in source and destination.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
