[
https://issues.apache.org/jira/browse/KAFKA-7500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16927978#comment-16927978
]
Ryanne Dolan commented on KAFKA-7500:
-------------------------------------
[~qihong] thanks for the questions.
> But couldn't find any consumer groups on dr2 related to consumer group
> test1grp.
MM2 does not create consumer groups for you or attempt to keep them in sync --
it only produces checkpoints (dr1.checkpoints.internal) that encode the state
of remote consumer groups. You must then do something with these checkpoints,
depending on your use-case. The RemoteClusterUtils class will read checkpoints
for you, which you can then use in interesting ways.
For example, you can use RemoteClusterUtils.translateOffsets() and the
kafka-consumer-groups.sh --reset-offsets tool to create a consumer group in dr2
based on MM2's checkpoints from dr1. Or, you can use RemoteClusterUtils in your
Consumer code to failover/failback automatically. Both require a bit of code,
but nothing too sophisticated.
Looking ahead a bit, this will be a ton easier when KIP-396 is merged
(KAFKA-7689). Once consumer offsets can be controlled from the Admin API, it
will be possible to consume checkpoints and update offsets directly. That will
enable the behavior you were expecting.
> By the way, how to set up and run this in a Kafka connect cluster?
MM2's Connectors are just plain-old Connectors. You can run them with
connect-standalone.sh or connect-distributed.sh as with any other Connector. To
do so, you need a worker config and a connector config as usual. The worker
config must include whatever client settings are required to connect to the
_target_ cluster (i.e. bootstrap servers, security settings), since the Worker
is what is actually producing downstream records. The connector configs, on the
other hand, need connection settings for _both_ source and target clusters
(e.g. source.cluster.bootstrap.servers, target.cluster.bootstrap.servers). The
Connectors use both source and target clusters when syncing topic configuration
etc.
There is an example Connector configuration here:
https://github.com/apache/kafka/pull/6295#issuecomment-522074048
> MirrorMaker 2.0 (KIP-382)
> -------------------------
>
> Key: KAFKA-7500
> URL: https://issues.apache.org/jira/browse/KAFKA-7500
> Project: Kafka
> Issue Type: New Feature
> Components: KafkaConnect, mirrormaker
> Affects Versions: 2.4.0
> Reporter: Ryanne Dolan
> Assignee: Manikumar
> Priority: Major
> Labels: pull-request-available, ready-to-commit
> Fix For: 2.4.0
>
> Attachments: Active-Active XDCR setup.png
>
>
> Implement a drop-in replacement for MirrorMaker leveraging the Connect
> framework.
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0]
> [https://github.com/apache/kafka/pull/6295]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
