Any help here would be greatly appreciated. On Sat, Aug 8, 2020, 12:13 PM Ananya Sen <ananya281...@gmail.com> wrote:
> Thank you Ryanne for the quick response. > I further want to clarify a few points. > > The mirror maker 2.0 is based on the Kafka Connect framework. In Kafka > connect we have multiple workers and each worker has some assigned task. To > map this to Mirror Maker 2.0, A mirror Maker will driver have some workers. > > 1) Can this number of workers be configured? > 2) What is the default value of this worker configuration? > 3) Does every topic partition given a new task? > 4) Does every consumer group - topic pair given a new task for replicating > offset? > > Also, consider a case where I have 1000 topics in a Kafka cluster and each > topic has a high amount of data + new data is being written at high > throughput. Now I want to set up a mirror maker 2.0 on this cluster to > replicate all the old data (which is retained in the topic) as well as the > new incoming data in a backup cluster. How can I scale up the mirror maker > instance so that I can have very little lag? > > On 2020/07/11 06:37:56, Ananya Sen <ananya281...@gmail.com> wrote: > > Hi > > > > I was exploring the Mirror maker 2.0. I read through this > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 > > documentation > > and I have a few questions. > > > > 1. For running mirror maker as a dedicated mirror maker cluster, the > > documentation specifies a config file and a starter script. Is this > mirror > > maker process distributed ? > > 2. I could not find any port configuration for the above mirror maker > > process, So can we configure mirror maker itself to run as a cluster > i.e > > running the process instance across multiple server to avoid downtime > due > > to server crash. > > 3. If we could somehow run the mirror maker as a distributed process > > then does that mean that topic and consumer offset replication will be > > shared among those mirror maker processes? > > 4. What is the default port of this mirror maker process and how can > we > > override it? > > > > Looking forward to your reply. > > > > > > Thanks & Regards > > Ananya Sen > > >
