As far as I know, there is no way for MM2 to be notified when a new topic is created, so it polls at a configurable interval. You can try adjusting the refresh.topics.interval.seconds property. It sounds like it's configured for 10 minutes.
Ryanne On Mon, Feb 20, 2023, 11:35 AM Fares Oueslati <oueslati.fa...@gmail.com> wrote: > Hello, > > I am currently using Kafka mirror maker v2 v3.2.0 to replicate a Kafka > cluster version 3.2.0 to another Kafka cluster version 3.2.0. However, I am > experiencing a delay issue where it takes around 10 minutes for a topic to > be created in the destination cluster after it has been created in the > source cluster. I am replicating all topics and groups with identity > replication and have provided the configs of the mirrors below. > > When I check the status, all the tasks are running smoothly and the logs > show no errors (I even increased the connect.root.logger.level to DEBUG). > The CPU and memory consumption of the cluster are both below the request > levels. > > Despite everything appearing to be fine, I am still experiencing a delay > when I create a topic in the source cluster. I should note that once the > topic is created, message replication is instantaneous, it is only the > topic creation that is delayed by a few minutes. > > Here are the configs for the different connectors of the MM2 cluster: > > mirrors: > - sourceConnector: > config: > consumer.auto.offset.reset: latest > sync.topic.acls.enabled: "false" > replication.policy.separator: "" > replication.policy.class: > org.apache.kafka.connect.mirror.IdentityReplicationPolicy > heartbeatConnector: > config: > heartbeats.topic.replication.factor: 1 > checkpointConnector: > tasksMax: 6 > config: > replication.policy.separator: "" > replication.policy.class: > org.apache.kafka.connect.mirror.IdentityReplicationPolicy > checkpoints.topic.replication.factor: 1 > sync.group.offsets.enabled: "true" > topicsPattern: ".*" > groupsPattern: ".*" > > > Thank you for your help on this. >
