Well, I don't know much about herders. If you can give some idea how to check it, I will try.
Peter On Fri, 20 Mar 2020 at 17:47, Ryanne Dolan <ryannedo...@gmail.com> wrote: > Hmm, that's weird. I'd expect the type of tasks to be evenly distributed as > well. Is it possible one of the internal topics are misconfigured s.t. the > Herders aren't functioning correctly? > > Ryanne > > On Fri, Mar 20, 2020 at 11:17 AM Péter Sinóros-Szabó > <peter.sinoros-sz...@transferwise.com.invalid> wrote: > > > I use tasks.max = 4. > > > > I see 4 tasks of MirrorSourceConnectors on MM2 instances A. > > I see 4 tasks of MirrorCheckpointConnector and 1 task of > > MirrorHeartbeatConnector on MM2 instance B. > > > > The number of tasks are well distributed, but the type of tasks are not. > > According to Connect documentation I expected 1-3 or 2-2 tasks of > > the MirrorSourceConnectors on the two MM2 instances. > > > > So is this a bug or an expected behaviour? > > > > Thanks, > > Peter > > > > On Fri, 20 Mar 2020 at 15:26, Ryanne Dolan <ryannedo...@gmail.com> > wrote: > > > > > Peter, in Connect the Connectors are only run on the leader node. Most > of > > > the work is done in the Tasks, which should be divided across nodes. > Make > > > sure you have tasks.max set to something higher than the default of 1. > > > > > > Ryanne > > > > > > On Fri, Mar 20, 2020, 8:53 AM Péter Sinóros-Szabó > > > <peter.sinoros-sz...@transferwise.com.invalid> wrote: > > > > > > > Hey, > > > > > > > > I am using MM2 to mirror A cluster to B with tasks.max = 4. > > > > > > > > I started two instances of MM2 and noticed that all > > > MirrorSourceConnectors > > > > were running in one instance and the rest of the connectors in the > > other. > > > > > > > > This results in a very uneven resource utilization and also it did > not > > > > really spread the mirroring oad between the two nodes. > > > > > > > > I assumed that MM2 will run 2-2 of those connectors in each instance. > > > > > > > > Is this current behaviour as expected or did I miss something on how > to > > > > configure it better? > > > > > > > > Thanks, > > > > Peter > > > > > > > > > >
