Ohhhh ok apologies. Interesting! On Tue, Oct 16, 2018 at 9:06 AM Jan Filipiak <jan.filip...@trivago.com> wrote:
> Hi Andrew, > > thanks for your message, you missed my point. > > Mirrormaker collocation with target is for sure correct. > But then group coordination happens across WAN which is unnecessary. > And I request to be thought about again. > I made a PR back then for zk Consumer to allow having 2 zookeeper > connects. One for group coordination one for broker and topic discovery. > > I am requesting this to be added to the kip so that the target cluster > can become the group coordinator. > > > > On 16.10.2018 15:04, Andrew Otto wrote: > >> I would generally say a LAN is better than a WAN for doing group > >> coordinaton > > > > For sure, but a LAN is better than a WAN for producing messages too. If > > there is network congestion during network production, messages will be > > dropped. With MirrorMaker currently, you can either skip these dropped > > messages, or have the MirrorMaker processes themselves die on produce > > failure, which will also cause (a series) of MirrorMaker consumer > > rebalances. > > > > > > > > > > > > > > On Tue, Oct 16, 2018 at 7:58 AM Jan Filipiak <jan.filip...@trivago.com> > > wrote: > > > >> Hi, > >> > >> Currently MirrorMaker is usually run collocated with the target cluster. > >> This is all nice and good. But one big obstacle in this was > >> always that group coordination happened on the source cluster. So when > >> then network was congested, you sometimes loose group membership and > >> have to rebalance and all this. > >> > >> So one big request from we would be the support of having coordination > >> cluster != source cluster. > >> > >> I would generally say a LAN is better than a WAN for doing group > >> coordinaton and there is no reason we couldn't have a group consuming > >> topics from a different cluster and committing offsets to another one > >> right? > >> > >> Other than that. It feels like the KIP has too much features where many > >> of them are not really wanted and counter productive but I will just > >> wait and see how the discussion goes. > >> > >> Best Jan > >> > >> > >> On 15.10.2018 18:16, Ryanne Dolan wrote: > >>> Hey y'all! > >>> > >>> Please take a look at KIP-382: > >>> > >>> > >> > https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0 > >>> > >>> Thanks for your feedback and support. > >>> > >>> Ryanne > >>> > >> > > >
