Sunny, check the groups and groups.blacklist properties. By default, MM2 won't replicate consumer groups from kafka-console-consumer, for example. Sometimes that confuses people when testing MM2.
Also check the logs to see if there is any reason MirrorCheckpointConnector might be failing to start. Ryanne On Tue, Aug 4, 2020 at 10:27 AM Sunny Lohani <sunny.loh...@gmail.com> wrote: > Hi Ryanne, > > First of all, thanks for a quick revert. Actually, I have a consumer > application consuming messages in cluster A and then I failover the > consumer from A to cluster B. Also, the consumer is subscribed to both > local and remote topics. Let's say the local topic name in both the > clusters A and B is test-topic. Here is a sequence of steps that I am > following before and after the failover: > > 1. Consumer application subscribes to topics test-topic and B.test-topic in > cluster A with group.id "test-consumer" > 2. It consumes some messages from both the topics. > 3. Now, we stop the consumer application and restart it pointing to cluster > B, subscribed to topics test-topic and A.test-topic, with group.id > "test-consumer". I use the RemoteClusterUtils.translateOffsets() here. > 4. The method returns an empty map as well as the checkpoints topic in both > the clusters is empty. > > Let me know if you see anything wrong here. > > Thanks, > Sunny > > > On Tue, Aug 4, 2020 at 8:26 PM Ryanne Dolan <ryannedo...@gmail.com> wrote: > > > Sunny, is it possible there are no consumer groups? There will be no > > checkpoints, and thus nothing to use for offset translation, if there are > > no upstream consumer groups. > > > > Ryanne > > > > On Tue, Aug 4, 2020, 9:28 AM Sunny Lohani <sunny.loh...@gmail.com> > wrote: > > > > > Hi, > > > > > > I have 2 data centers, each having single node Zookeeper and Kafka > > cluster. > > > I have a topic (single partition) in both the data center kafka > > clusters. I > > > am using MM 2.0 as a dedicated cluster for bi-directional replication > of > > > the topic as well as using RemoteClusterUtils.translateOffsets() in my > > > application for offset translation during failover. But the method is > > > returning an empty map due to which the consumer is not resuming from > > > proper offsets for local/remote topics. > > > > > > When I investigated further, I found that the checkpoint > > > topics A.checkpoints.internal and B.checkpoints.internal in respective > > > clusters do not have any kafka message. I don't see any errors in the > > > mirror maker console logs. I searched everywhere on the internet but > > could > > > not get any help. Below is the mm.properties: > > > > > > clusters = A, B > > > A.bootstrap.servers = 10.34.45.113:19092 > > > B.bootstrap.servers = 10.34.45.113:29092 > > > > > > A->B.enabled = true > > > A->B.topics = .* > > > B->A.enabled = true > > > B->A.topics = .* > > > > > > # Setting replication factor of newly created remote topics > > > replication.factor=1 > > > > > > checkpoints.topic.replication.factor=1 > > > heartbeats.topic.replication.factor=1 > > > offset-syncs.topic.replication.factor=1 > > > > > > offset.storage.replication.factor=1 > > > status.storage.replication.factor=1 > > > config.storage.replication.factor=1 > > > > > > sync.topic.acls.enabled = false > > > > > > emit.checkpoints.enabled = true > > > emit.checkpoints.interval.seconds = 5 > > > ---- > > > > > > Need help on this urgently. Thanks in advance. > > > > > > Thanks & Regards, > > > Sunny Kumar Lohani, > > > > > >
