Hi, Hope you all are safe and well!
I found success in using Mirrormaker2 to sync data from source A (1 broker in a 3node cluster) to target B(1 broker in a 3node cluster) where I was running mirrormaker2 in the controller node in cluster B. Now, I see an issue with the sync for a topic whose size in cluster A is approximately 332GB but the same topic in cluster B is hardly 8KB. And this issues is not seen for any other topics as they are getting sync'ed as expected.. Could you please help me understand why this topic is failing to get sync'ed Source cluster A ============= kafka-topics --zookeeper zknode:2181 --describe --topic bigtopic Topic: bigtopic PartitionCount: 1 ReplicationFactor: 3 Configs: cleanup.policy=delete Topic: bigtopic Partition: 0 Leader: 1002 Replicas: 1002,1001,1003 Isr: 1003,1002,1001 du -sh /kafkadata/logs/bigtopic-0/ 331G /kafkadata/logs/bigtopic-0/ Target cluster B ============ kafka-topics --zookeeper zknodeB:2181 --describe --topic bigtopic Topic: bigtopic PartitionCount: 1 ReplicationFactor: 3 Configs: cleanup.policy=delete Topic: bigtopic Partition: 0 Leader: 6 Replicas: 6,4,5 Isr: 6,4,5 du -sh /kafkadata/logs/bigtopic-0/ 8.0K /kafkadata/logs/bigtopic-0/ And here is the mm2.properties that I use- # mm2.properties clusters = A, B A.bootstrap.servers = Ahostname:9092 B.bootstrap.servers = Bhostname:9092 A->B.enabled = true A->B.topics = somanytopicnames_including_bigtopic replication.factor=3 refresh.topics.interval.seconds = 10 refresh.groups.interval.seconds = 10 sync.topic.config.enabled = true replication.policy.separator = source.cluster.alias = target.cluster.alias = #replication.policy.class= io.strimzi.kafka.connect.mirror.IdentityReplicationPolicy groups = .* tasks.max=100 emit.checkpoints.interval.seconds = 10 security.protocol=SASL_PLAINTEXT sasl.mechanism=PLAIN sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \ username="CONFIDENTIAL" \ password="vvvvvvv"; Thanks & Regards, Rijo Roy
