With 3 nodes, I assume you mean 3 clusters? If I understand correctly, say you have 3 clusters, A, B, and C, you simultaneously: - want to copy from A and B to C, to get an aggregation in C - want to copy fram A and C to B, to get a fail-back aggregation in B. Now what will happen when a message is produced in cluster a? - it will be copied to both C and B. - the copy wil cause a new copy in C and B, etc. There are several ways out if this, depending on your use case. It's pretty easy to change the behaviour of the mirrormaker, for example to copy it to $topic-aggregation instead of $topic, and to not copy it when the topic ends with aggregation
On Tue, Jun 28, 2016 at 10:15 AM cs user <acldstk...@gmail.com> wrote: > Hi All, > > So I understand I can run the following to aggregate topics from two > different clusters into a mirror: > > bin/kafka-run-class.sh kafka.tools.MirrorMaker --consumer.config > sourceCluster1Consumer.config --consumer.config > sourceCluster2Consumer.config --num.streams 2 --producer.config > targetClusterProducer.config --whitelist=".*" > > Lets say my kafka mirror cluster consists of 3 nodes, can the above process > be started on each of the 3 nodes, so that in the event it fails on one the > other 2 will keep going? > > Or should only one of the nodes attempt to perform the aggregation? > > Thanks! >
