Hi Kafka team,

Hope you're doing well. I'm investigating how to use the MirrorMaker 2
source connector to replicate data between different Kafka clusters. One
scenario is that in the source Kafka cluster, we have multiple topics, then
we expect all messages from these topics will be sent to one topic in the
target Kafka. That is, all messages from multiple topics will be aggregated
to one topic.

We plan to use transformation to implement the feature.
org.apache.kafka.connect.transforms.RegexRouter (
https://docs.confluent.io/kafka-connectors/transforms/current/regexrouter.html#description)
can update the record's topic using the configured regular expression and
replacement string, which can be used to change the original topics to the
same name. Here's the example:

```
transforms=RenameTopicSimple
transforms.RenameTopicSimple.type=org.apache.kafka.connect.transforms.RegexRouter
transforms.RenameTopicSimple.regex=.*
transforms.RenameTopicSimple.repalcement=target-kafka-topic-name
```

My question is:
1. Can *configurations above* +  *MirrorMaker 2 source connector *meet
our requirements?
2. Is there any potential issues when messages from multiple topics in
different source Kafka clusters are aggregated to one topic of the target
Kafka?

Best,
Ran Qin

Reply via email to