David Dufour created KAFKA-15102:
------------------------------------
Summary: Mirror Maker 2 - KIP690 backward compatibility
Key: KAFKA-15102
URL: https://issues.apache.org/jira/browse/KAFKA-15102
Project: Kafka
Issue Type: Bug
Components: mirrormaker
Affects Versions: 3.1.0
Reporter: David Dufour
According to KIP690, "When users upgrade an existing MM2 cluster they don’t
need to change any of their current configuration as this proposal maintains
the default behaviour for MM2."
In particular, the naming of the topic {{mm2-offset-syncs}} was supposed to use
fixed separators:
{{ }}{{default}} {{String offsetSyncsTopic(String targetAlias) { }}{{return}}
{{" mm2-offset-syncs."}} {{+ targetCluster + }}{{{}".internal"{}}}{{{};}{}}}
The implementation is actually slightly different:
[https://github.com/apache/kafka/blob/400d39bb0efaa4fdda4c0597a63a1ddd17082291/connect/mirror-client/src/main/java/org/apache/kafka/connect/mirror/DefaultReplicationPolicy.java#L84]
Instead, the separator is subject to customization.
As a consequence, when an upgrade is performed, if the separator is customized
with replication.policy.separator, the name of this internal topic changes. It
then generates issues like:
Caused by: java.util.concurrent.ExecutionException:
org.apache.kafka.common.errors.InvalidTopicException: Topic
'mm2-offset-syncs_bkts28_internal' collides with existing topics:
mm2-offset-syncs.bkts28.internal
It has been observed that the replication can then be broken sometimes several
days after the upgrade (reason not identified). By deleting the old topic name,
it recovers.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
