Zhaoli created KAFKA-14807:
------------------------------
Summary: MirrorMaker2 config
source.consumer.auto.offset.reset=latest leading to the pause of replication of
consumer groups
Key: KAFKA-14807
URL: https://issues.apache.org/jira/browse/KAFKA-14807
Project: Kafka
Issue Type: Bug
Components: mirrormaker
Affects Versions: 3.3.2, 3.3.1, 3.4.0
Environment: centos7
Reporter: Zhaoli
We use MirrorMaker2 to replicate messages and consumergroup offsets from kafka
cluster `source` to cluster `target`.
In order to reduce the load on the source cluster, we add this configuration to
mm2 to avoid replicate the whole history messages:
{code:java}
source.consumer.auto.offset.reset=latest {code}
After that, we found part of the consumergroup offsets have stopped replicating.
The common characteristic of these consumergroups is their EMPTY status，which
means they have no active members at that monent. All the active
consumergroups‘ offset replication work as normal.
After researching the source code，we found this is because the configuration
above also affect the consumption of topic `mm2-offset-syncs`, therefore the
map `
offsetSyncs` dosen't hold the whole topicPartitions, and the lost
topicPartitions lead to the pause of replication of the EMPTY consumer groups.
{code:java}
private final Map<TopicPartition, OffsetSync> offsetSyncs = new HashMap<>();
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
