Hi hudeqi, Thank you for your comments! Related to the topic list: you are correct, the partition metrics are created eagerly, so even if a topic has no active traffic, the metrics are visible. I missed this fact when creating the KIP. For the group list: the group metrics are created lazily, so for the groups, the metrics are only created if there was a checkpoint replicated (i.e. the group is included in the replication AND has committed offset for at least one of the replicated partitions).
Regardless of the eager/lazy nature of these metrics, I think that having a centralized way of getting the list of replicated topics/groups still makes sense, and it can simplify access to this information for external clients. E.g. We could add a listener implementation which writes the list of topics/groups into a compact topic, which then can be consumed by external clients, which is simpler than scraping partition metrics and collecting the topic tags. I will add corrections to the Rejected Alternatives section with more details about the metrics. Thanks, Daniel hudeqi <16120...@bjtu.edu.cn> ezt írta (időpont: 2023. ápr. 19., Sze, 4:22): > Hi, I have some questions about motivation. > The problem to be solved by this kip is that it cannot accurately monitor > the topic and group currently being replicated? From my point of view, > using MirrorSourceMetrics.recordRate can monitor the configured topic list > that is currently being replicated (even if the topic has no data), and > using MirrorCheckpointMetrics.CHECKPOINT_LATENCY can monitor the currently > replicated group list (if it is wrong, please correct me). > > best, > hudeqi > > "Dániel Urbán" <urb.dani...@gmail.com>写道： > > Hello everyone, > > > > I would like to bump this KIP. Please consider reviewing it, as it would > > improve the monitoring capabilities around MM2. > > I also submitted a PR (https://github.com/apache/kafka/pull/13595) to > > demonstrate the current state of the KIP. > > > > Thanks in advance, > > Daniel > > > > Dániel Urbán <urb.dani...@gmail.com> ezt írta (időpont: 2023. ápr. 13., > Cs, > > 15:02): > > > > > Hi everyone, > > > > > > I would like to start a discussion on KIP-918: MM2 Topic And Group > > > Listener ( > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-918%3A+MM2+Topic+And+Group+Listener > > > ). > > > This new feature of MM2 would allow following the latest set of > replicated > > > topics and groups, which is currently not possible in MM2. > Additionally, > > > this would help IdentityReplicationPolicy users, as they could use > this new > > > feature to track the replicated topics (which is not available through > the > > > policy due to topics not being renamed during replication). > > > > > > Thanks in advance, > > > Daniel > > > >
