[
https://issues.apache.org/jira/browse/KAFKA-14837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17704472#comment-17704472
]
hudeqi commented on KAFKA-14837:
--------------------------------
Regarding this issue, I have a direct and easy way: In the
"refreshConsumerGroups" of MirrorCheckpointConnector, call
"listConsumerGroupOffsets" for each group, then filter according to topicFilter
and groupFilter to get the list of groups to be synchronized, trigger rebalance
according to the increase or decrease of this group list, and finally set The
filtered groups is distributed to all tasks for execution.
In this way, MirrorCheckpointConnector avoids the rabalance interference caused
by the addition and deletion of irrelevant groups (KAFKA-14837), and reduces
the rpc calls of "listConsumerGroupOffsets" in each task about irrelevant
groups (KAFKA-14842).
> The MirrorCheckPointConnector of MM2 will rebalance frequently, when the
> source cluster group is many more and changes frequently (but the list of
> configured synchronous group does not change)
> ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-14837
> URL: https://issues.apache.org/jira/browse/KAFKA-14837
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect, mirrormaker
> Affects Versions: 3.3.2
> Reporter: hudeqi
> Assignee: hudeqi
> Priority: Major
>
> In practice, I found that when I configure a mirror checkpoint connector,
> because the source cluster has a large number of group or the number of group
> under a topic changes frequently, the connector will frequently rebalance
> between its tasks, although there is no change in the synchronized group list
> of the configuration.
> I don't think connector should rebalance frequently in this case to affect
> group synchronization tasks without any group changes.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
