Hi Andrew,
thanks for your message, you missed my point.
Mirrormaker collocation with target is for sure correct.
But then group coordination happens across WAN which is unnecessary.
And I request to be thought about again.
I made a PR back then for zk Consumer to allow having 2 zookeeper
connects. One for group coordination one for broker and topic discovery.
I am requesting this to be added to the kip so that the target cluster
can become the group coordinator.
On 16.10.2018 15:04, Andrew Otto wrote:
I would generally say a LAN is better than a WAN for doing group
coordinaton
For sure, but a LAN is better than a WAN for producing messages too. If
there is network congestion during network production, messages will be
dropped. With MirrorMaker currently, you can either skip these dropped
messages, or have the MirrorMaker processes themselves die on produce
failure, which will also cause (a series) of MirrorMaker consumer
rebalances.
On Tue, Oct 16, 2018 at 7:58 AM Jan Filipiak <jan.filip...@trivago.com>
wrote:
Hi,
Currently MirrorMaker is usually run collocated with the target cluster.
This is all nice and good. But one big obstacle in this was
always that group coordination happened on the source cluster. So when
then network was congested, you sometimes loose group membership and
have to rebalance and all this.
So one big request from we would be the support of having coordination
cluster != source cluster.
I would generally say a LAN is better than a WAN for doing group
coordinaton and there is no reason we couldn't have a group consuming
topics from a different cluster and committing offsets to another one
right?
Other than that. It feels like the KIP has too much features where many
of them are not really wanted and counter productive but I will just
wait and see how the discussion goes.
Best Jan
On 15.10.2018 18:16, Ryanne Dolan wrote:
Hey y'all!
Please take a look at KIP-382:
https://cwiki.apache.org/confluence/display/KAFKA/KIP-382%3A+MirrorMaker+2.0
Thanks for your feedback and support.
Ryanne
