[
https://issues.apache.org/jira/browse/KAFKA-18125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17901904#comment-17901904
]
Jarkko Jaakola commented on KAFKA-18125:
----------------------------------------
The PR can be backported to previous Kafka versions. The changes should apply
cleanly at least to 3.7, 3.8 and 3.9.
> Implement MirrorCheckpointConnector max task limit
> --------------------------------------------------
>
> Key: KAFKA-18125
> URL: https://issues.apache.org/jira/browse/KAFKA-18125
> Project: Kafka
> Issue Type: Improvement
> Components: mirrormaker
> Reporter: Jarkko Jaakola
> Priority: Major
>
> Mirrormaker 2 MirrorCheckpointConnector currently can create as many tasks as
> globally configured. Primary limiting factor for number of tasks up to the
> configured maximum is the number of known consumer groups. On larger mirrored
> environments where known consumers groups is large this leads to excessive
> amount of checkpoint tasks.
> As the offset sync topic is single partition topic there will be excessive
> amount of tasks reading from and writing to this topic. The targeted Kafka
> broker can and will be under heavy burden.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
