Thanks Ryanne for your quick reply. I have reached out the KIP owner for status and pending work on this. He has created draft PR also for this and you have reviewed it. https://github.com/apache/kafka/pull/8604/files
I did launch a sample consumer but still the MBeans were not created. I will debug this further and update you. I have one more question regarding putting DC source information in message header. I have seen lot of people have mentioned this in discussion thread of the KIP. If I understand correctly, adding additional information in message header has following Pros and Cons: Pros: 1. Single topic for consumers to consume the data and no changes at consumer end (interested in processing all data) in case of multi DC setup. Cons: 1. Consumers interested in consuming produced at particular DC (lets say DC1), would need to process all data and filter out the records based on message header. This would lead to unnecessary consumption of disk/network resources. Please let me know If I have misunderstood this. Thanks, Nitin On Wed, Jun 17, 2020 at 4:09 AM Ryanne Dolan <ryannedo...@gmail.com> wrote: > Nitin, there is a recently accepted KIP to add the formatters here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-597%3A+MirrorMaker2+internal+topics+Formatters > . > Not > sure if there is a JIRA or PR for the KIP yet. > > MirrorCheckpointConnector MBeans won't get created until there are > checkpoints written, which only happens when there are consumer groups to > replicate. Try creating a consumer group that matches the groups whitelist. > > Hope that helps! > > Ryanne > > On Tue, Jun 16, 2020 at 8:45 AM nitin agarwal <nitingarg...@gmail.com> > wrote: > > > Hi, > > > > I have a few questions regarding MirrorMaker 2.0. > > > > Q1. Is there JIRA for adding HeartbeatMessageFormatter and > > CheckpointMessageFormatter? I want to read messages of the > > topic source.checkpoints.internal. > > > > Q2. While running MirrorMaker 2.0, I couldn't find the MBeans > > for MirrorCheckpointConnector. There is no error in logs regarding this > and > > tasks regarding this conector are running fine. I see following messages > in > > logs: > > [2020-06-16 19:06:41,210] INFO > > WorkerSourceTask{id=MirrorHeartbeatConnector-0} Finished commitOffsets > > successfully in 2 ms > > (org.apache.kafka.connect.runtime.WorkerSourceTask:523)[2020-06-16 > > 19:06:41,210] INFO WorkerSourceTask{id=MirrorCheckpointConnector-0} > > Committing offsets > > (org.apache.kafka.connect.runtime.WorkerSourceTask:424)[2020-06-16 > > 19:06:41,210] INFO WorkerSourceTask{id=MirrorCheckpointConnector-0} > > flushing 0 outstanding messages for offset commit > > (org.apache.kafka.connect.runtime.WorkerSourceTask:441) > > > > Thanks, > > Nitin > > >
