no worries, glad i could clarify
On 16.10.2018 15:14, Andrew Otto wrote: > Ohhhh ok apologies. Interesting! > > On Tue, Oct 16, 2018 at 9:06 AM Jan Filipiak <jan.filip...@trivago.com> > wrote: > >> Hi Andrew, >> >> thanks for your message, you missed my point. >> >> Mirrormaker collocation with target is for sure correct. >> But then group coordination happens across WAN which is unnecessary. >> And I request to be thought about again. >> I made a PR back then for zk Consumer to allow having 2 zookeeper >> connects. One for group coordination one for broker and topic discovery. >> >> I am requesting this to be added to the kip so that the target cluster >> can become the group coordinator. >> >> >> >> On 16.10.2018 15:04, Andrew Otto wrote: >>>> I would generally say a LAN is better than a WAN for doing group >>>> coordinaton >>> >>> For sure, but a LAN is better than a WAN for producing messages too. If >>> there is network congestion during network production, messages will be >>> dropped. With MirrorMaker currently, you can either skip these dropped >>> messages, or have the MirrorMaker processes themselves die on produce >>> failure, which will also cause (a series) of MirrorMaker consumer >>> rebalances. >>> >>> >>> >>> >>> >>> >>> On Tue, Oct 16, 2018 at 7:58 AM Jan Filipiak <jan.filip...@trivago.com> >>> wrote: >>> >>>> Hi, >>>> >>>> Currently MirrorMaker is usually run collocated with the target cluster. >>>> This is all nice and good. But one big obstacle in this was >>>> always that group coordination happened on the source cluster. So when >>>> then network was congested, you sometimes loose group membership and >>>> have to rebalance and all this. >>>> >>>> So one big request from we would be the support of having coordination >>>> cluster != source cluster. >>>> >>>> I would generally say a LAN is better than a WAN for doing group >>>> coordinaton and there is no reason we couldn't have a group consuming >>>> topics from a different cluster and committing offsets to another one >>>> right? >>>> >>>> Other than that. It feels like the KIP has too much features where many >>>> of them are not really wanted and counter productive but I will just >>>> wait and see how the discussion goes. >>>> >>>> Best Jan >>>> >>>> >>>> On 15.10.2018 18:16, Ryanne Dolan wrote: >>>>> Hey y'all! >>>>> >>>>> Please take a look at KIP-382: >>>>> >>>>> >>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 >>>>> >>>>> Thanks for your feedback and support. >>>>> >>>>> Ryanne >>>>> >>>> >>> >> >
