Hi Julia, Sounds like KAFKA-9981 [1]. This is a known issue with MirrorMaker 2 that impacts horizontal scalability and has not yet been addressed. There is some work in progress to fix this issue [2], but the effort hasn't received much attention to date. There may be other issues as well, but until KAFKA-9981 is addressed, running MirrorMaker 2 in a multi-node cluster will be at best difficult and worst, impossible.
[1] - https://issues.apache.org/jira/browse/KAFKA-9981 [2] - https://cwiki.apache.org/confluence/display/KAFKA/KIP-710%3A+Full+support+for+distributed+mode+in+dedicated+MirrorMaker+2.0+clusters Cheers, Chris On Wed, Mar 23, 2022 at 11:15 AM Kalimova, Julia <julia.kalim...@blackrock.com.invalid> wrote: > > > Hello! > > > > I’m wondering if it’s possible to scale up MM2 by running multiple > instances per data center? > > For scalability purposes, I would like to run 2 instances of > connect-mirror-maker.sh in one data center. However, I cannot get 2 > instances of mirror maker to work at the same time: once I start up the > second mirror maker, it takes over for the first one, and the first one > completely stops replicating: i.e. rather than scaling up and rebalancing, > the whole workload is still handled by a single instance. > > What am I missing here? Would greatly appreciate any guidance with this! > Thank you! > > > This message may contain information that is confidential or privileged. > If you are not the intended recipient, please advise the sender immediately > and delete this message. See > http://www.blackrock.com/corporate/compliance/email-disclaimers for > further information. Please refer to > http://www.blackrock.com/corporate/compliance/privacy-policy for more > information about BlackRock’s Privacy Policy. > > > For a list of BlackRock's office addresses worldwide, see > http://www.blackrock.com/corporate/about-us/contacts-locations. > > © 2022 BlackRock, Inc. All rights reserved. >
