Daniel Florek created KAFKA-13988:
-------------------------------------
Summary: Mirrormaker 2 auto.offset.reset=latest not working
Key: KAFKA-13988
URL: https://issues.apache.org/jira/browse/KAFKA-13988
Project: Kafka
Issue Type: Bug
Environment: Source Kafka cluster running on Ubuntu 20
Source Kafka cluster Kafka v0.10
Target Kafka cluster running in AWS MSK
Target Kafka cluster Kafka v2.6.2
Mirrormaker version 3.2.0 running on Ubuntu 20.
Reporter: Daniel Florek
Hi.
I have problem setting up mirroring with MM2 from latest offset between 2
clusters. In logs I can see that Consumer that is consuming topics has
auto.offset.reset property set to latest. But still topics are read from offset
0. I am using following configuration:
{code:java}
clusters = A, B
A.bootstrap.servers = broker-01A:9092
B.bootstrap.servers = broker-01B:9092,broker-02B:9092,broker-03B:9092
replication.policy.class =
org.apache.kafka.connect.mirror.IdentityReplicationPolicy
#Enable replication between clusters and define topics which should be
replicated
A->B.enabled = true
A->B.topics = .*
A->B.replication.factor=3
A->B.emit.heartbeats.enabled = true
A->B.emit.checkpoints.enabled = true
auto.offset.reset=latest
consumer.auto.offset.reset=latest
A.consumer.auto.offset.reset=latest
B.consumer.auto.offset.reset=latest
refresh.topics.enabled=true
heartbeats.topic.replication.factor=1
checkpoints.topic.replication.factor=1
offset-syncs.topic.replication.factor=1
config.storage.replication.factor = 1
offset.storage.replication.factor = 1
status.storage.replication.factor = 1 {code}
I am using Kafka 3.2.0 for Mirrormaker 2. Source kafka cluster is 1 broker
running on EC2 instance in AWS (quite an old version I think 0.10). Target
kafka cluster contains 3 brokers running in AWS MSK (version 2.6.2).
Could you point me what I am doing wrong? Or is this possibly a bug?
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
